USING  MULTIPLE  ROBUST  PARAMETER  DESIGN  TECHNIQUES 
TO  IMPROVE  HYPERSPECTRAL  ANOMALY  DETECTION 
ALGORITHM  PERFORMANCE 

THESIS 

Matthew  Davis,  Captain,  USAF 
AFIT  /  GOR/ENS/ 09-05 


DEPARTMENT  OF  THE  AIR  FORCE 
AIR  UNIVERSITY 

AIR  FORCE  INSTITUTE  OF  TECHNOLOGY 


Wright-Patterson  Air  Force  Base,  Ohio 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


The  views  expressed  in  this  thesis  are  those  of  the  author  and  do  not  reflect  the 
official  policy  or  position  of  the  United  States  Air  Force,  Department  of  Defense  or 
the  United  States  Government. 


AFIT  /  GOR/ENS  /  09-05 


USING  MULTIPLE  ROBUST  PARAMETER  DESIGN 
TECHNIQUES  TO  IMPROVE  HYPERSPECTRAL 
ANOMALY  DETECTION  ALGORITHM 
PERFORMANCE 


THESIS 


Presented  to  the  Faculty 
Department  of  Operational  Sciences 
Graduate  School  of  Engineering  and  Management 
Air  Force  Institute  of  Technology 
Air  University 

Air  Education  and  Training  Command 
in  Partial  Fulfillment  of  the  Requirements  for  the 
Degree  of  Master  of  Science  in  Operations  Research 

Matthew  Davis,  B.S. 

Captain,  USAF 

March  2009 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED. 


AFIT  /  GOR/ENS  /  09-05 


USING  MULTIPLE  ROBUST  PARAMETER  DESIGN 
TECHNIQUES  TO  IMPROVE  HYPERSPECTRAL 
ANOMALY  DETECTION  ALGORITHM 
PERFORMANCE 

Matthew  Davis,  B.S. 

Captain,  USAF 


Approved: 


Kenneth  W.  Bauer,  Jr.,  Ph.D.  Date 

Chairman 


J.O.  Miller,  Ph.D. 
Member 


Date 


AFIT  /  GOR/ENS  /  09-05 


Abstract 

Detecting  and  identifying  objects  of  interest  is  the  goal  of  all  remote  sensing. 
New  advances,  specifically  in  hyperspectral  imaging  technology  have  provided  the 
analyst  with  immense  amounts  of  data  requiring  evaluation.  Several  filtering  tech¬ 
niques  or  anomaly  detection  algorithms  have  been  proposed.  However,  most  new 
algorithms  are  insufficiently  verified  to  be  robust  to  the  broad  range  of  hyperspec¬ 
tral  data  being  made  available.  One  such  algorithm,  AutoGAD,  is  tested  here  via 
two  separate  robust  parameter  design  techniques  to  determine  optimal  parameters 
for  consistent  performance  on  a  range  of  data  with  large  attribute  variances.  Ad¬ 
ditionally,  the  results  of  the  two  techniques  are  compared  for  overall  effectiveness. 
The  results  of  the  test  as  well  as  optimal  parameters  for  AutoGAD  are  presented 
and  future  research  efforts  proposed. 
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USING  MULTIPLE  ROBUST  PARAMETER  DESIGN 
TECHNIQUES  TO  IMPROVE  HYPERSPECTRAL  ANOMALY 
DETECTION  ALGORITHM  PERFORMANCE 


1.  Introduction 

The  overarching  goal  of  all  remote  sensing,  regardless  of  the  specific  technology 
employed,  is  the  ability  to  detect  and  identify  objects  of  interest  within  a  relatively 
large  area  while  maintaining  some  measure  of  separation  between  the  sensor  and 
the  area  of  interest.  The  increasing  availability  and  quality  of  hyperspectral  remote 
sensing  technology  has  created  a  need  for  ways  to  quickly  and  accurately  detect 
and  identify  objects  of  interest  within  a  rapidly  growing  library  of  hyperspectral 
data.  This  research  seeks  to  improve  upon  the  speed,  accuracy,  and  robustness 
of  autonomous  detection  methods  previously  proposed  in  the  technical  literature. 
Robust  parameter  design  is  a  technique  well  suited  to  optimizing  existing  detection 
methods. 

1.1  Background 

The  past  two  decades  have  seen  the  development  of  various  methods  for  target 
detection  in  hyperspectral  imagery  (HSI).  The  motivations  for  these  methods  are  as 
varied  as  the  methods  themselves,  but  the  problem  is  essentially  universal;  find  a 
small  number  of  unique  objects  scattered  across  a  relatively  large  area  [12].  Regard¬ 
less  of  the  specific  object  of  interest,  HSI  and  its  subsequent  analysis  has  proven  an 
effective  tool  for  extracting  object  location  and  type  from  an  area  of  interest. 

Historically,  analysis  methods  have  framed  the  problem  of  target  detection  in 
mainly  two  ways.  The  first  approach  uses  the  unique  spectral  signatures  of  the  mate- 
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rials  within  the  image  and  compares  them  to  known  reference  signatures  in  order  to 
identify  an  object  [13].  Most  of  the  time,  however,  reference  signatures  are  unavail¬ 
able  and  some  other  methodology  must  be  employed  to  detect  and  identify  objects 
of  interest.  This  leads  to  the  second  approach  proposed  in  the  literature.  In  this  ap¬ 
proach,  the  spectral  signatures  of  pixels  within  an  image  are  compared  to  the  other 
pixels  within  the  image.  Anomalous  pixels  are  identified  by  their  relative  nonhomo¬ 
geneity  with  respect  to  the  other  pixels  in  the  image.  This  particular  approach  can 
be  further  divided  into  two  separate  methodologies  that  will  be  explained  in  greater 
detail  in  Ch.  2. 

1 . 2  Methodology 

The  idea  of  detecting  anomalous  pixels  within  an  image  without  a  prioi  knowl¬ 
edge  of  targets’  unique  spectral  signatures  is  a  more  appropriate  approach  to  the 
problem  faced  by  the  Department  of  Defense.  Within  the  military  context,  detec¬ 
tion  and  identification  of  targets  takes  on  added  complexity  in  that  targets  ’do  not 
wish  to  be  found’  and  will  typically  be  concealed  within  the  background  in  some 
way.  The  methods  for  concealment  are  varied,  but  appropriate  application  of  target 
detection  techniques  to  hyperspectral  data  can  locate  all  but  the  most  well  hidden 
targets.  Additional  constraints  are  placed  on  the  DoD  problem  instance  by  the  need 
for  fast  solutions.  Johnson  [5]  highlights  this  with  his  assertion  that  some  future 
evolution  of  the  autonomous  global  anomaly  detector  (AutoGAD)  algorithm  could 
potentially  be  run  by  a  UAV’s  processing  software  and  provide  information  on  po¬ 
tential  targets  in  real  time.  This  kind  of  real  time  detection  is  critical  to  meaningful 
employment  of  HSI  technology  at  the  tactical  and  operational  level. 

With  any  new  algorithm  or  approach  to  solving  a  problem,  the  quality  of 
the  method  should  be  evaluated  solely  on  observed  performance.  In  order  to  make 
quantitative  assessments,  the  solution  methodology  should  be  tested  on  problem 
instances  with  known  solutions  so  that  comparisons  to  known  optimal  values  can 
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be  made.  Based  on  this  comparison  to  ground  truth,  performance  statistics  are 
developed.  These  performance  statistics  are  then  used  to  evaluate  the  quality  of  the 
proposed  method  relative  to  existing  methods  and  their  performance  statistics  on 
the  same  problem  instances. 

While  this  practice  is  widely  used  in  the  literature,  Hooker  [4]  suggests  that 
this  competitive  testing  approach  leads  researchers  to  tweak  the  parameters  of  their 
algorithm  to  outperform  others  on  the  test  problems.  This  over-specification  of  the 
algorithm  can  lead  to  very  poor  performance  on  a  broader  set  of  real  world  problems. 

The  anomaly  detection  algorithm,  AutoGAD,  developed  by  Johnson  [5]  was 
shown  to  work  well  for  the  four  images  used  during  initial  testing.  However,  as  with 
most  algorithms,  similarities  between  the  development  and  test  data,  combined  with 
limited  test  data  led  to  a  situation  where  the  algorithm  is  known  to  work  very  well 
for  data  similar  to  the  development  and  test  data,  but  its  solution  quality  for  other 
data  types  is  unknown. 

One  alternative  to  this  competitive  testing  is  using  Design  of  Experiments 
(DOE)  to  evaluate  the  performance  of  an  algorithm.  DOE  provides  a  methodology 
allowing  the  researcher  to  make  statistically  sound  inferences  about  algorithm  per¬ 
formance  over  a  range  of  problem  instances  using  a  relatively  small  number  of  test 
runs.  A  subset  of  the  DOE  framework,  Robust  Parameter  Design  (RPD),  is  a  general 
method  developed  by  Taguchi  in  the  early  1980s  that  emphasizes  choosing  the  levels 
of  controllable  parameters  to:  (1)  ensure  that  the  mean  of  the  output  response  is  at 
a  desired  level  and  (2)  ensure  that  the  variability  around  this  target  value  is  as  small 
as  possible  [8]. 

Framing  anomaly  detection  using  AutoGAD  as  a  RPD  problem  facilitates  the 
selection  of  optimal  user  defined  parameters  within  AutoGAD  making  the  algorithm 
robust  to  a  wide  variety  of  image  types.  Additionally,  these  parameters  reduce 
the  variability  of  AutoGAD’s  performance.  Therefore,  the  result  is  twofold;  the 
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optimized  parameters  allow  AutoGAD  to  perform  anomaly  detection  not  only  well, 
but  more  importantly,  perform  consistently  over  a  wide  variety  of  image  types. 

1.3  Research  Objectives 

This  research  will  employ  robust  parameter  design  to  develop  additional  testing 
scenarios  for  AutoGAD  in  order  to  determine  is  applicability  and  effectiveness  over 
a  wide  range  of  image  types.  Additionally,  optimal  settings  for  the  multiple  user 
adjustable  parameters  present  in  the  algorithm  will  be  determined. 

The  remainder  of  this  thesis  is  organized  as  follows.  Chapter  2  contains  a 
discussion  on  the  basics  of  HSI,  a  more  detailed  description  of  the  detection  method¬ 
ologies  proposed  in  the  current  literature,  and  a  discussion  of  robust  parameter 
design  based  on  a  review  the  literature.  Chapter  3  describes  how  robust  parame¬ 
ter  design  was  specifically  applied  to  AutoGAD  including  a  detailed  description  of 
the  experiment  performed.  Chapter  4  includes  the  results  of  the  experiment  and 
the  resultant  optimized  algorithm  parameters.  Chapter  5  summarizes  the  work  and 
suggests  directions  for  future  research. 
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2.  Literature  Review 


2.1  HSI  Basics 

HSI  is,  at  the  most  basic  level,  a  digital  photograph.  Like  a  digital  photo,  HSI 
is  composed  of  picture  elements,  or  pixels,  which  divide  the  two  dimensional  space  of 
the  scene  into  smaller  regions.  The  pixels  create  a  grid  which  can  be  indexed  in  both 
dimensions  to  form  a  referencing  structure  allowing  each  pixel  within  an  image  to  be 
individually  referenced.  In  a  standard  digital  photo,  each  pixel  contains  information 
about  a  portion  of  the  electromagnetic  (EM)  spectrum,  namely  the  visible  portion. 
The  visible  portion,  can  be  subdivided  into  bands,  for  instance  the  red,  green,  and 
blue.  Each  pixel  is  assigned  an  intensity  level  within  each  band,  the  combination  of 
which  determines  the  overall  color  or  reflectance  signature  of  the  pixel.  HSI  is  based 
on  these  same  ideas,  however,  it  differs  from  ordinary  photos  in  several  ways. 

The  first,  and  most  important  difference  between  standard  digital  photographs 
and  HSI  is  the  number  of  bands.  Where  standard  photos  can  be  represented  in  as 
few  as  one  band  (black  and  white  photos),  HSI  is  composed  of  anywhere  from  tens 
to  hundreds  of  contiguous  bands  [12],  The  second  difference  between  standard  pho¬ 
tography  and  hyperspectral  imaging  is  the  range  of  EM  energy  represented  within 
a  single  image.  While  standard  photos  only  represent  energy  from  the  visible  wave¬ 
lengths,  HSI,  in  general,  contains  contiguous  data  from  mid-wave  infra-red  to  ultra¬ 
violet  wavelengths.  Figure  2.1  highlights  the  portion  of  the  EM  spectrum  used  by 
HSI  sensors. 

Now  that  a  basic  analogy  for  HSI  has  been  drawn  from  standard  photography, 
The  reader  should  be  familiarized  with  how  the  data  from  an  HSI  image  is  organized 
for  processing  and  analysis.  HSI  data  can  be  thought  of  as  a  cube  or  3-dimensional 
matrix  of  data.  The  front  face  of  the  cube  is  defined  by  the  spatial  dimensions  of 
the  image.  Viewing  the  data  from  this  orientation  is  analogous  to  looking  at  the 
image  printed  on  a  page  or  displayed  on  a  screen.  The  third  dimension  of  the  data 
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Figure  2.1  Electromagnetic  Spectrum  [6] 

represents  the  spectral  dimension  of  the  data.  This  dimension  can  be  thought  of  as 
a  stack  of  images,  each  capturing  a  very  narrow  piece  of  the  scene’s  EM  reflectance 
spectra.  Figures  2.2  and  2.3  illustrate  this  idea.  According  to  these  definitions,  each 
cell  within  the  data  cube  can  now  be  indexed,  first  by  its  two  spatial  dimension 
values,  and  then  by  its  spectral  dimension.  Each  cell  can  now  be  referenced  by  a 
coordinate  triple,  (■ i,j ,  k),  where  i  and  j  represent  the  spatial  coordinates  of  a  pixel 
within  the  image  and  the  value  within  the  cell  is  the  pixel’s  reflectance  response  in 
the  kth  spectral  band. 

Visualizing  the  data  as  a  cube  is  useful  for  understanding  where  the  data  comes 
from  and  how  it  is  organized  spatially,  but  maintaining  the  data  in  this  format  is  not 
useful  for  analysis.  The  most  efficient  structure  for  doing  the  various  manipulations 
is  a  two-dimensional  matrix.  Transforming  the  data  to  this  format  is  straightforward 
and  will  be  illustrated  by  example.  Let  C  be  the  data  from  a  hyperspectral  image  100 
pixels  wide  by  150  pixels  long  and  containing  200  contiguous  spectral  bands  resulting 
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Figure  2.2  Dimensions  of  Hyperspectral  Data  Cube  [11] 


in  15000  pixels  each  containing  200  elements  of  spectral  data.  Each  of  these  15000 
200  by  1  pixel  vectors  can  then  be  made  a  column  in  a  new  matrix.  This  new  matrix 
is  200  rows  by  15000  columns  and  each  column  contains  the  spectral  data  of  a  single 
pixel  from  the  original  image.  The  transformation  process  is  visualized  in  Figure  2.4 


2.2  Approaches  to  HSI  Analysis 

There  are  currently  two  major  approaches  to  finding  objects  of  interest.  One 
approach  taken  by  some  is  to  develop  or  obtain  ground  truth  signatures  for  particu¬ 
lar  objects  or  materials  of  interest  then  compare  this  true  signature  to  the  signature 
of  each  pixel  within  an  image.  Pixels  whose  signatures  match  the  ground  truth 
signature  to  within  some  statistical  threshold  are  nominated  as  potential  target  lo¬ 
cations.  While  this  technique  seems  logical,  there  are  several  situations  where  this 
methodology  breaks  down.  The  first  and  most  frequent  problem  encountered  when 
employing  this  technique  in  an  autonomous  way  is  the  lack  of  ground  truth  signature 


2-3 


Figure  2.3  Spectral  Layers  of  Hyperspectral  Data  Cube  [7] 

data  for  the  objects  within  a  particular  scene  that  may  be  of  interest.  Secondly,  high 
spectral  resolution  sensors  are  significantly  impacted  by  environmental  effects  at  the 
time  the  image  of  interest  is  created.  Time  of  year,  time  of  day,  relative  humidity, 
and  imaging  angle  are  just  a  few  of  the  factors  that  effect  how  much  and  what  type 
of  reflectance  data  reach  the  sensor  during  a  given  image  capture  sequence.  Since 
these  environmental  factors  vary  through  time,  reflectances  from  the  materials  of  a 
particular  area  can  vary  greatly  from  one  image  to  the  next. 

These  obstacles  make  the  development  of  a  robust  autonomous  signature  match¬ 
ing  algorithm  difficult.  The  other  family  of  detection  methodologies  is  referred  to 
as  Anomaly  Detection.  This  approach  can  be  subdivided  further  into  two  types; 
distribution  based  anomaly  detectors  (local  or  global)  that  assume  a  distribution 
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Data  cube: 


Figure  2.4  Transforming  a  HSI  data  cube  into  a  data  matrix  [7] 

(typically  normal)  of  the  pixel  vectors,  and  global  linear  mixture  model  detectors 
[5], 

Local  anomaly  detectors  consider  each  pixel  individually  comparing  its  re¬ 
flectance  values  to  the  mean  and  covariance  values  of  its  neighboring  pixels.  Mul¬ 
tivariate  statistical  tests  are  used  to  declare  the  pixel  an  outlier  with  respect  to  its 
neighborhood.  The  definition  of  the  neighborhood  is  unique  to  each  instance  of  this 
type  of  detector,  but  is  usually  8~50  of  the  pixel’s  nearest  neighbors.  These  types 
of  models  are  usually  referred  to  as  local  normal  models  since  they  assume  Gaussian 
distributions  of  the  pixel  vectors  [5]. 
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The  other  type  of  distribution  based  detectors  are  referred  to  as  global  normal 
mixture  models.  These  models  assume  an  image  contains  some  number  C  of  material 
types,  or  endmembers.  These  endmembers  each  have  a  specific  spectral  signature 
within  the  image.  Each  pixel  is  then  assumed  to  be  composed  of  a  random  vector 
having  one  of  several  multivariate  Gaussian  (normal)  distributions.  The  probability 
density  function  (pdf)  of  the  scene  is  a  Gaussian  mixture  distribution: 

c  c 

g(x)  =  ncN(x\uc ,  £g);  ttc  >  0;  ^  vrc  =  1  (2.1) 

C—  1  C=1 

where  nc  is  the  probability  of  belonging  to  endmember  c  [15].  This  methodology 
is  effective,  but  Smetek  [13]  notes  that  no  guidance  is  available  for  determining  the 
number  of  endmembers  \C\  in  a  particular  image. 

The  remaining  methodology  is  an  approach  known  as  global  linear  mixture 
model  anomaly  detection.  Like  global  normal  mixture  models,  this  approach  as¬ 
sumes  the  image  contains  some  deterministic  number  of  endmembers,  C.  However, 
the  global  linear  mixture  model  assumes  each  pixel’s  total  reflectance  is  linear  com¬ 
bination  of  the  reflectances’  of  each  of  the  endmembers  present  in  the  image.  The 
coefficients  of  each  of  the  C  terms  in  the  linear  combination  that  makes  up  a  par¬ 
ticular  pixel’s  total  reflectance  can  be  interpreted  as  the  fractional  abundance  of  the 
endmembers  within  that  pixel  [15]. 

This  global  linear  mixture  approach  is  used  by  Johnson  [5]  to  develop  Anto- 
GAD.  Since  the  subject  of  this  research  is  not  to  develop  a  new  approach,  but  rather 
to  optimize  the  application  of  Johnson’s  algorithm,  more  rigorous  development  of 
these  methodologies  are  omitted. 
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2.3  Overview  of  Auto  GAD 


The  AutoGAD  algorithm  developed  by  Johnson  [5],  is  a  multi-step  approach  to 
assisting  imagery  analysts  by  identifying  potential  targets  within  raw  hyperspectral 
image  data.  There  are  several  user-adjustable  parameters  built  in  to  the  algorithm 
and  these  will  be  central  to  the  Robust  Parameter  Design  problem  to  be  developed 
later.  However,  in  order  to  make  these  parameters  meaningful  to  the  reader,  an 
overview  of  AutoGAD  is  appropriate. 

The  reader  should  note  that  the  following  is  entirely  a  paraphrase  of  Johnson’s 
work  and  as  such  will  only  be  referenced  here  [5],  however,  his  references  to  other 
works  are  repeated  where  appropriate. 

Recalling  section  2.2,  the  initial  step  of  any  HSI  analysis  tool  is  to  prepare 
the  data  for  analysis  by  converting  it  from  a  3-dimensional  cube  to  a  2-dimensional 
matrix.  Once  this  conversion  is  complete,  the  detection  algorithm  consists  of  four 
phases.  These  phases  are  depicted  in  the  flow  diagram  in  Figure  2.5. 


Figure  2.5  Process  Flow  for  Target  Detection  [5] 


The  first  step,  Feature  Exraction  I,  involves  dimensionality  reduction  and 
whitening  of  the  data.  Dimensionality  is  typically  reduced  using  Principal  Compo¬ 
nents  Analysis  (PGA).  PGA  is  a  widely  used  analysis  technique  that  projects  data 
into  a  subspace  whose  dimensions  maximize  the  variance  of  the  data.  The  dimen¬ 
sions  of  this  new  subspace  are  ordered  according  the  variance  of  the  data  captured 
by  each  dimension  or  principal  component.  Thus,  the  first  principal  component  con¬ 
tains  the  most  variance,  the  second  principal  component  contains  the  second  most 
variance  and  so  on.  Once  the  data  is  projected  into  this  new,  variance-ordered  space, 
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it  is  whitened.  Whitening  of  the  data  simply  means  that  the  data  is  centered  and 
scaled  so  that  is  has  mean  0  and  unit  variance. 

Now  that  the  data  has  been  whitened  in  a  projection  space  where  variance  is 
maximized,  the  amount  of  variance  to  be  retained  must  be  determined.  This  number 
of  dimensions  and  subsequent  amount  of  variance  to  be  retained  is  determined  by 
Johnson’s  Maximum  Distance  Secant  Line  (MDSL)  method  [5].  The  proper  dimen¬ 
sionality  suggested  by  may  be  adjusted  by  the  user  to  consistently  keep  more  or  less 
dimensions  than  the  technique  suggests  as  appropriate. 

The  second  step,  Feature  Extraction  11,  utilizes  a  piece  of  software  called  Fas- 
tlCA  to  conduct  Independent  Component  Analysis  (ICA).  Further  projecting  the 
data  into  independent  components  rather  than  the  uncorrelated  ones  that  result 
from  PCA  serves  to  comply  with  the  assumption  of  the  Linear  Mixture  Model  (LMM) 
that  the  components  are,  in  fact,  independent.  The  LMM  is  one  of  several  ways  of 
characterizing  imagery  data  where  multiple  types  of  objects,  each  with  their  own 
distinct  spectral  signatures,  make  up  the  spatial  region  of  a  scene.  For  instance,  one 
particular  image  may  contain  grass,  dirt,  concrete,  rocks,  and  vehicles.  Each  of  these 
would  have  a  distinct  spectral  signature  and  ideally  each  of  their  signatures  would 
be  represented  by  a  dimension  of  the  data  after  ICA  is  performed.  If  this  occurs, 
then  when  each  of  the  components  is  spatially  plotted,  one  image  shows  only  grass, 
another  only  concrete,  another  only  the  vehicles,  and  so  on.  Thus,  the  vehicles, 
presumably  of  interest  to  the  analyst,  have  been  isolated  from  the  rest  of  the  image. 

Feature  Selection,  the  third  step,  takes  the  data  plotted  on  the  components 
generated  by  FastICA  and  determines  which  components  or  features  within  the  data 
have  the  potential  to  be  targets.  It  does  this  by  exploiting  the  assumption  that 
targets  are  rare  and  small  within  the  scene.  If  these  properties  hold  for  the  targets 
within  an  image,  the  number  of  pixels  that  have  high  a  percentage  of  a  target’s 
spectral  signature  will  be  small,  while  the  number  of  pixels  that  contain  a  high 
percentage  of  non-target  signatures  will  be  significantly  higher.  By  plotting  the 
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frequency  histogram  of  pixel  scores  for  a  particular  dimension  and  distinguishing 
targets  from  background  by  finding  the  first  score  bin  without  any  pixels,  a  portion 
of  the  pixels  can  be  classified  as  background  while  the  remaining  are  classified  as 
potential  targets.  Placing  these  two  values  in  ratio,  a  Potential  Target  Signal  to 
Noise  ratio  is  created.  Thus,  a  noise  floor  for  SNR  can  be  set  and  applied  to  the 
by-component  histogram  plots.  Any  plot  with  an  SNR  below  the  noise  floor  is 
determined  to  be  representative  of  a  non-target  spectral  signature.  Plots  with  SNR’s 
above  the  noise  floor  represent  potential  target  signatures. 

The  fourth  and  final  step,  Identification,  reexamines  the  plots  with  SNR’s 
above  the  noise  floor  and  the  high  scoring  pixels  within  those  plots  are  considered  as 
targets.  Pixels  whose  scores  are  very  close  to  zero  bin  cut  line  have  high  potential 
for  misclassihcation.  To  mitigate  this  misclassihcation  potential,  Johnson  employed 
an  amplification  technique  called  Iterative  Adaptive  Noise  Filtering.  This  process 
iteratively  multiplies  any  score  above  the  cut  line  by  a  number  greater  than  1  while 
scores  below  the  cut  line  are  multiplied  by  a  number  smaller  than  one.  After  many 
replications,  all  pixels  scores  are  well  separated  and  visual  representations  of  these 
new  scores  are  near  black  and  white  when  plotted  on  a  gray  scale.  This  gives  potential 
targets  high  contrast  from  the  background  improving  the  designation  capability  of 
AutoGAD. 

2.4  Robust  Parameter  Design 

In  almost  all  processes,  whether  manufacturing,  transportation,  or  anomaly 
detection,  process  managers  desire  some  optimized  process  output  that  varies  as  lit¬ 
tle  as  possible  between  products,  shipments,  or  images.  Robust  Parameter  Design 
(RPD)  is  a  method  for  reducing  process  or  product  variation  developed  by  Genichi 
Taguchi  and  introduced  in  the  United  States  in  the  early  1980’s.  While  Taguchi’s 
philosophy  was  met  with  great  interest  and  was  implemented  by  several  large  cor¬ 
porations,  his  methodology  became  a  source  of  contention  within  the  community  of 


2-9 


statisticians  and  quality  engineers.  The  questionable  analysis  techniques  Taguchi 
suggested  were  unacceptable  to  some  and  as  a  result,  the  response  surface  approach 
has  been  suggested  as  a  collection  of  tools  that  allow  Taguchi’s  concepts  to  be  ap¬ 
plied  using  a  more  statistically  sound  and  efficient  approach  to  analysis  [10].  Both 
approaches  will  be  used  during  this  research,  and  each  will  be  developed  in  more 
detail  in  the  following  sections. 

In  general,  RPD  seeks  to  simultaneously  produce  some  target  response  from 
the  system  while  minimizing  the  variance  of  that  response.  These,  at  times  com¬ 
peting,  objectives  are  achieved  by  adjusting  settings  of  controllable  factors  to  both 
manipulate  the  value  of  the  response  and  dampen  the  effect  of  uncontrollable,  or 
noise  factors  within  the  system.  RPD  takes  advantage  of  this  two  class  variable 
structure  and  correlation  between  the  classes  to  locate  optimized  settings.  The  two 
classes  of  variables  are  control  variables  and  noise  variables.  Control  variables  are 
those  variables  in  a  process  that  remain  unchanged  once  selected,  therefore  adding 
no  variation.  Noise  variables  are  factors  that  are  difficult  to  control  during  normal 
process  operation  and  may  change  without  warning  [2], 

A  critical  piece  of  the  variable  structure  is  correlation  between  at  least  some  of 
the  control  and  noise  variables.  If  all  the  noise  variables  are  completely  independent 
of  the  control  variables,  then  their  variance  will  remain  constant  across  the  entire 
range  of  the  control  variables.  If  however,  some  interaction  between  control  and 
noise  variables  exists,  then  some  point  or  set  of  points  in  the  control  space  produces 
minimum  variance  across  the  range  of  noise  variables.  This  point  is  well  illustrated 
in  Figure  2.6,  where  the  change  in  response  across  the  range  of  noise  from  low  to 
high  is  minimized  when  the  control  is  set  to  high. 

Given  this  interaction  between  control  and  noise  factors,  controls  can  be  clas¬ 
sified  into  at  least  one  of  three  groups,  otherwise  they  are  insignificant  and  removed 
from  consideration.  The  first  classification  of  control  factors  is  determined  by  their 
influence  on  the  response.  If  control  factors  influence  a  change  in  response  as  they 
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Figure  2.6  Control-by-Noise  Interaction  Plot  [2] 

move  across  their  range,  then  they  are  said  to  be  location  factors.  Control  factors 
that  interact  with  noise  variables  are  called  dispersion  factors.  Finally,  control  fac¬ 
tors  that  are  location  factors  (affect  mean  process  response),  but  are  not  dispersion 
factors  (do  not  interact  with  noise  factors)  are  called  adjustment  factors  [2], 

2. 4-1  Crossed  Array  Designs  —  The  Taguchi  Approach 

The  use  of  orthogonal  arrays  in  experimental  design  is  central  to  the  methodol¬ 
ogy  suggested  by  Taguchi.  Specifically,  an  orthogonal  array  for  the  control  variables 
called  the  inner  array  is  crossed  with  an  orthogonal  array  for  the  noise  variables 
called  the  outer  array  [10].  An  example  experiment  of  this  type  is  illustrated  in 
Figure  2.7.  In  this  example  a  22  inner  array  is  crossed  with  a  22  outer  array.  The 
resultant  16-run  experiment  consists  of  4  repetitions  of  the  outer  array;  one  at  each 
of  the  inner  array  design  points.  Designs  of  this  type  are  not  necessarily  full  factorial 
in  nature.  Reduced  resolution  designs  can,  and  often  are,  used  in  both  the  inner  and 
outer  arrays.  However,  the  outer  array  design,  whatever  its  size,  must  be  completed 
at  each  of  the  inner  array  design  points. 

Taguchi  suggested  that  the  data  collected  from  each  outer  array  could  be  rolled 
into  a  single  summary  statistic  providing  information  about  the  mean  and  variance. 
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For  instance,  in  Figure  2.7  each  dot  represents  an  observation,  so  each  block  of  4 
dots  would  have  its  own  summary  statistic  which  Taguchi  calls  a  signal-to-noise  ratio 
(SNR)  [10]. 

SNR  is  a  broad  term  defined  in  myriad  ways  by  authors  across  a  wide  variety 
of  technical  disciplines.  However,  Taguchi  suggests  4  primary  definitions  for  SNR. 
He  categorizes  them  by  the  optimality  conditions  defined  for  the  response  of  interest: 

Smaller  is  better  —  Optimality  is  a  minimized  response 

Larger  is  better  —  Optimality  is  a  maximized  response 

Target  is  best  —  Optimality  is  a  response  equal  to  some  target  value 

Before  beginning  a  discussion  of  the  types  of  problems  and  the  SNR’s  Taguchi 
proposed  to  help  solve  them,  it  is  important  to  determine  how  potential  robust  solu¬ 
tions  will  be  compared  to  each  other.  It  is  intuitive  to  simply  compute  the  difference 
between  the  response  of  the  current  solution  and  the  optimal  response  and  use  these 
differences  to  rank  potential  solutions.  However,  in  cases  where  multiple  potential 
solutions  may  exist,  some  smaller  and  some  larger  than  the  optimal  one,  simply 
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computing  the  difference  by  subtracting  the  optimal  solution  from  the  potential  re¬ 
sponses  may  lead  negative  distance  to  optimality.  Thus,  a  squared  distance  function, 
known  as  a  quadratic  loss  function  is  used  to  evaluate  the  quality  of  potential  so¬ 
lutions.  The  generalized  quadratic  loss  function  expressed  in  terms  of  the  expected 
value  of  the  response  is  given  in  Equation  2.2  where  L  is  the  square  of  the  distance 
between  the  expected  value  of  the  response  and  the  target  value. 


L  =  E,(y  -  t )2 


(2.2) 


2-4- 1-1  Smaller  is  Better 

In  the  case  where  a  smaller  response  is  better  the  target  value  of  the  response,  t  is 
assumed  to  be  zero.  Thus,  Eq  2.2  reduces  to  L  =  Ez(y)2  and  the  SNR  for  this  case 
is  given  below  in  Equation  2.3. 

n  2 

SNR,  =  -10  log  V  —  (2.3) 

n 

t=i 

This  formulation  for  SNR  results  from  a  need  to  find  control  variable  values  that 
minimize  the  expected  square  error  Ez(y)2,  where  Ez  is  the  expectation  of  response 
across  the  noise  variable  distribution.  Thus,  the  SNR  is  computed  by  summing  the 
square  errors  observed  in  each  outer  array  and  dividing  that  sum  by  the  number 
of  observations  or  points  in  the  outer  array,  n.  An  SNR  is  then  computed  at  each 
design  point  in  the  inner  array  and  maximized,  since  the  —10  log  transformation  is 
used  [10]. 


2-13 


2. 4. 1.2  Larger  is  Better 


In  problems  where  maximization  of  the  response  is  the  goal,  y  in  Eq  2.2  is  replaced 
by  1  jy.  This  allows  L  to  approach  zero  as  the  value  of  y  increases  and,  again  the 
expected  square  error  reduces  to  Ez(l/y)2.  The  SNR  for  problems  where  larger 
response  is  better  is  given  in  Equation  2.4.  Just  as  before,  control  variable  settings 
that  maximize  the  SNR  are  best  [10]. 

n  ..  /  2 

SNR;  =  -10  log  —  (2.4) 

i=  1 


2.4- 1-3  Target  is  Best 

In  this  instance,  the  control  variables  are  adjusted  to  bring  the  response  to  some 
target  level.  This  can  be  thought  of  as  the  Goldilocks  case:  ‘Not  too  hot,  not  too 
cold,  but  just  right’  [14].  Taguchi  proposed  two  different  SNRs  for  this  case  with  the 
nature  of  the  system  determining  which  is  proper. 

The  first  system  to  be  considered  is  one  where  the  mean  and  variance  of  the 
response  can  be  altered  independently.  Because  of  this  independence,  a  two  step 
analysis  is  employed  to  locate  on  target  and  robust  control  factors.  Taguchi  suggests 
as  a  first  step  in  the  analysis  to  adjust  the  controls  that  affect  mean  response,  called 
tuning  factors,  such  that  Ez(y)  =  t.  Adjusting  these  tuning  factors  has  no  effect 
on  variance,  so  the  SNR  remains  unchanged  during  this  step.  The  second  step  is 
adjust  the  remaining  factors  such  that  the  SNR  is  maximized.  If  the  first  step  is 
successful,  the  tuning  factors  are  set  such  that  y  —  t  and  the  square  error  loss 
function,  Ez(y  —  t)2,  reduces  to  Var (y),  thus  the  SNR  is  given  by  Equation  2.5, 
where  s 2  is  the  sample  variance  of  the  responses  in  each  outer  array.  Once  again, 
the  —10  log  transformation  is  used  so  that  minimizing  s 2  maximizes  SNR  [10]. 
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(2.5) 


SNRTl  =  -10  logs2 

The  second  type  of  target-is-best  system  is  one  where  response  variance  is 
related  to  response  mean.  This  relationship  is  hopefully  linear,  but  nonetheless 
prevents  the  independent  adjustment  of  tuning  factors  to  bring  the  mean  on  target. 
If  the  mean  variance  relationship  is  linear,  a  coefficient  of  variation  is  easily  computed 
by  <Jy/iiy  The  two  step  analysis  can  again  be  accomplished  by  adjusting  tuning 
factors  to  bring  the  mean  on  target  while  leaving  the  variation  coefficient  unchanged. 
The  SNR  is  now  computed  using  the  Equation  2.6,  where  y2  /  s2  is  the  squared  inverse 
of  the  sample  coefficient  of  variation  [10]. 


SNRTa  =  lOlog0^  (2.6) 

For  a  variety  of  reasons,  Taguchi’s  suggested  SNRs  described  above  are  criti¬ 
cized  in  later  literature.  First,  with  the  exception  of  the  SNR  in  Eq  2.6,  they  contain 
some  transformation  of  the  units  of  the  response  of  interest.  Despite  these  ques¬ 
tionable  formulations,  the  critique  most  often  encountered  in  the  literature  is  the 
confounding  of  mean  and  variance  into  a  single  measure.  Many  authors  propose  the 
development  of  separate  models  for  mean  and  variance [1,  9,  16].  The  motivation  be¬ 
ing  to  gain  better  understanding  of  the  process  itself  and  thus  be  able  to  make  better 
decisions  about  where  tradeoffs  between  mean  and  variance  of  the  response  can  be 
made.  A  final  problem  with  the  confounding  of  mean  and  variance  is  that  multiple 
control  factor  setting  combinations  may  produce  identical  SNRs,  thus  the  interpre¬ 
tation  of  a  particular  SNR  is  unclear  with  regard  to  the  specific  control  settings  that 
produced  it  [10]. 
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2-4-2  Combined  Array  Designs  -  Response  Surface  Methods 

One  alternative  the  previously  mentioned  authors  propose  is  the  use  of  re¬ 
sponse  surface  methodology  (RSM),  which  has  become  widespread  in  solving  RPD 
problems.  RSM  gives  the  analyst  the  ability  to  develop  a  designed  experiment  so 
the  relationship  between  controls,  noise,  and  response  can  be  modeled.  Typically,  a 
second  order  response  surface  model  is  developed.  This  provides  linear  effects  from 
controls  and  noise,  as  well  as  quadratic  effects  from  controls,  and  interaction  effects 
between  controls  and  noise.  The  general  form  of  the  model  is  given  by  Eq.  2.7. 

?/(x,  z )  =  A)  +  x'/3  +  x'Bx  +  z'y  +  x'Az  +  e  (2.7) 

In  Eq.  2.7,  /3  and  7  are  vectors  of  coefficients  for  control  and  noise  main 
effects,  B  is  a  matrix  of  coefficients  of  the  quadratic  control  effects,  A  is  a  matrix 
of  control-by-noise  interaction  coefficients,  and  pure  error  is  given  by  e;  ~  A(0,(T2). 
The  variance  of  Eq.  2.7  is  now  easily  obtained  by  treating  the  noise  z  as  a  random 
variable  [9]  and  applying  the  variance  operator  to  Eq  2.7.  The  variance  equation  is 

Var  2[r/(x,  z)]  =  (7'  +  x'A)Var2(z)(7'  +  x'A)'  +  a2,  (2.8) 

where  Varz( z)  is  the  variance-covariance  matrix  of  z.  Expanding  this  equation  for 
multiple  noise  factors  ( Zi,Zj ),  yields  the  following  result, 


Var 2[r/(x,  zu  zf)]  =(7 *  +  xA?:)2Var(zj)  +  (7,-  +  xAi)2Var(^) 

+  2(7*  +  xAj)(7j  +  xAj)Cov(^j,  zf)  +  a2.  (2.9) 


2. 4-2.1  Types  of  Noise 

Eq.  2.9  above  maintains  the  treatment  of  z  as  a  random  variable.  However,  there  are 
a  couple  assumptions  made  about  noise  when  it  is  known  to  be  a  continuous  variable 
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that  help  to  reduce  the  complexity  of  the  variance  computation  suggested  by  Eq. 
2.9.  The  first  and  broadest  assumption  is  that  certain  attributes  of  the  distribution 
of  z  are  known.  Specifically,  the  variance-covariance  matrix  of  z  is  assumed  to  be 
known.  Additionally,  it  is  often  assumed  that  the  covariance  between  noise  variables 
is  zero.  Taking  these  assumptions  into  account  the  new  variance  model  is  given  by 
Eq.  2.10.  [2] 


Var z[y(x,  zh  Zj)\  =  (7 *  +  xA.4)2Var(^)  +  (7 j  +  xAi)2Var(^-)  +  cr2  (2.10) 

If  however,  the  noise  variables  are  categoric  instead  of  continuous,  the  assump¬ 
tions  made  above  can  be  avoided.  In  general,  making  less  assumptions  provides  a 
more  accurate  analysis.  Generating  a  variance  model  with  categorical  noise  variables 
is  not  difficult,  especially  since  any  number  of  categorical  variables  with  any  number 
of  categories  can  always  be  reduced  to  a  single  categorical  variable  with  rz  categories, 
where  rt  is  the  number  of  categories  of  the  ith  of  n  noise  variables,  and 

n 

r*=nr<-1  (2-n) 

i=i 

where  1  is  subtracted  to  account  for  a  baseline  category. 

Since  all  noise  is  now  captured  within  a  single  categorical  variable  with  multiple 
categories,  the  probability  of  occurrence  for  each  category  can  be  computed.  In 
most  cases,  the  probabilities  of  each  categorical  noise  factor  are  known  a  priori,  and 
if  they  aren’t,  a  uniform  distribution  is  a  reasonable  assumption  since  it  provides 
equal  opportunity  for  each  type  of  noise  and  thus  a  uniformly  robust  solution  should 
result.  These  probabilities,  P (Category  i)  =  pi,  have  a  multinomial  distribution,  thus 
the  variance-covariance  matrix  is  fully  characterized  as  long  as  the  p.i  s  are  known. 
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Using  dummy  variables: 


{1,  Category  =  i 
0,  otherwise 

the  variance-covariance  matrix  is  given  by 


i  =  l,2,...,rz,  (2.12) 


Pi(l  -  Pi) 

~P\P2 

~PlPrz 

Varz(z)  = 

-P2P\ 

p2(l-p2) 

~P2Prz 

(2.13) 

PrzPl 

~PrzP2 

^3 

1 

to 

The  test  range  for  continuous  noise  variables  is  generally  selected  such  that 
the  center  of  the  range  is  the  estimated  mean  and  the  high  and  low  levels  are  ±1  or 
±2  standard  deviations  from  the  mean.  These  values  are  set  prior  to  conducting  the 
experiment,  thus  failing  to  account  for  observed  process  variation  and  inducing  error 
in  the  analysis.  Additionally,  if  more  than  one  noise  variable  exists,  any  covariance 
between  them  is  assumed  to  be  zero,  inducing  even  more  error  in  the  model.  These 
error  inducing  assumptions  that  must  be  made  to  conduct  continuous  noise  analysis 
are,  in  the  categorical  noise  case,  traded  for  a  single,  much  less  risky  assumption 
that  must  only  be  made  in  rare  cases  where  the  pis  for  each  category  cannot  be 
computed.  Furthermore,  when  the  assumption  of  equal  probability  must  be  made, 
it  leads  to  conservative  analysis  in  that  it  provides  a  solution  that  is  robust  across 
all  noise  categories. 
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3.  Methodology 

The  previous  chapter  gave  a  brief  overview  of  the  AutoGAD  algorithm  and  two  ap¬ 
proaches  to  RPD.  This  chapter  will  begin  with  a  description  of  the  input  variables 
from  AutoGAD  that  will  serve  as  the  control  and  noise  space  and  give  a  short  descrip¬ 
tion  of  the  Measures  of  Performance  used  to  quantify  response.  Then  a  description 
of  how,  using  the  two  approaches,  RPDs  are  developed  for  AutoGAD. 

3. 1  Inputs 

As  described  in  Ch.  2,  the  AutoGAD  algorithm  is  rather  complex  and  highly 
customizable.  This  end-user  flexibility  creates  myriad  sources  of  response  adjustment 
as  well  as  a  large  control-noise  interaction  space.  Table  3.1  lists  both  the  control  and 
noise  input  parameters. 

The  first  control  factor,  Dimension  Adjust,  allows  the  user  to  keep  more  or  less 
dimensions  for  analysis  than  suggested  by  the  MDSL  technique  employed  during  the 
FE  I  phase.  The  second  control,  Max  Score  Threshold,  is  the  cut  line  for  raw  IC  pixel 
scores  above  which  a  pixel  becomes  a  candidate  to  be  a  potential  target.  Bin  Width 
SNR  determines  the  bin  width  used  during  the  zero-detection  histogram  method 
for  determining  which  IC  maps  contain  targets.  PT  SNR  Threshold  is  similar  to 
Max  Score  Threshold,  but  is  on  the  SNR  scale  and  is  used  only  on  IC’s  nominated 
as  containing  targets.  It  is  used  to  decide  which  pixels  have  a  high  enough  SNR 
to  actually  be  targets.  Likewise,  Bin  Width  Identify,  determines  bin  width  during 
the  zero-detection  histogram  procedure  during  the  Identification  phase.  Smooth 
Iterations  High  and  Low  determine  the  number  of  iterations  of  the  IAN  filtering 
process  for  low  and  high  SNR  objects  respectively.  Low  SNR  is  the  decision  point 
between  low  and  high  SNR.  The  ninth  control,  Window  Size,  determines  the  window 
size  used  for  the  IAN  Filtering  subroutine.  Both  the  tenth  and  eleventh  controls  are 
binary,  on/off,  controls.  Threshold  both  sides  makes  a  decision  about  considering 
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Input  Parameter 

Type 

Classification 

Test  Range 

Dimension  Adjust 

Discrete 

Control 

[-2.  2] 

Max  Score  Threshold 

Continuous 

Control 

(6.  14] 

Bin  Width  SNR 

Continuous 

Control 

[0.01,  0.1] 

PT  SNR  Threshold 

Continuous 

Control 

[1.6] 

Bin  Width  Indentify 

Continuous 

Control 

[0.01,  0.1] 

Smooth  Iterations  High 

Discrete 

Control 

[50,  150] 

Smooth  Iterations  Low 

Disrete 

Control 

[5,  45] 

Low  SNR 

Continuous 

Control 

[4,  14] 

Window  Size 

Discrete 

Control 

[1.9] 

Threshold  Both  Sides 

Categorical 

Control 

[0,  1] 

Clean  Signal 

Categorical 

Control 

[0,  1] 

Image 

Categorical 

Noise 

[1,  2,  3,  4,  5,  6,  7,  8] 

Table  3.1  Control  and  Noise  Factors  and  their  ranges 


pixels  with  large  negative  IC  scores  as  potential  targets.  On  (1),  considers  any  pixel 
with  large  magnitude  score,  while  off  (0)  only  considers  pixels  in  the  direction  of  the 
largest  magnitude  score.  Finally,  Clean  Signal  turns  the  IAN  Filtering  subroutine 
on  and  off. 

For  this  analysis,  the  single  noise  factor  considered  is  the  image  data  presented 
to  the  algorithm.  While  other  noise  variables  exist  within  this  problem  instance, 
they  are  for  the  most  part  controllable  relative  to  the  presumably  infinite  space  of 
possible  image  data  that  may  be  encountered  during  a  real  world  implementation 
of  this  algorithm.  Additionally,  during  this  analysis  the  noise  space  was  limited  to 
8  images  taken  from  single  set  of  image  data.  Similarities  in  the  image  data  may 
lead  to  over-specification  of  the  controls  from  a  global  perspective,  but  the  data  set 
is  sufficient  to  illustrate  the  technique. 


3. 2  Outputs 

While  the  single  product  of  the  AutoGAD  algorithm  is  a  scene  map  identifying 
potential  target  locations,  this  product  is  not  directly  quantifiable.  Therefore,  4 
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Output  Parameter 

Units 

Range 

Time 

Seconds 

[0,  oo] 

True  Positive  Fraction 

N/A 

[0.  1] 

False  Positive  Fraction 

N/A 

[0.  1] 

Target  Fraction  Percent 

Percent  (%) 

[0,  100] 

Table  3.2  Measures  of  Performance  and  their  ranges 


performance  parameters  related  to  target  prediction  map  were  collected  and  used  to 
quantify  algorithm  performance. 

The  first  and  most  intuitive  parameter  is  time,  measured  in  seconds  from  the 
instantiation  to  completion  of  the  algorithm.  The  second  parameter  is  True  Positive 
Fraction  (TPF)  and  is  based  on  a  comparison  of  AutoGAD  predictions  to  ground 
truth.  TPF  is  computed  by  dividing  the  number  of  pixels  AutoGAD  correctly  calls 
targets  by  the  number  of  target  pixels  in  the  ground  truth  image.  False  Positive 
Fraction  (FPF)  is  the  third  measure  of  performance  and  is  computed  by  dividing 
the  number  of  non-target  pixels  erroneously  declared  targets  by  the  algorithm  by  the 
number  of  non-target  pixels  in  the  ground  truth  data.  The  final  measure  of  algorithm 
performance  is  Target  Fraction  Percent  (TFP).  TFP  is  simply  the  ratio  of  correct 
target  declarations  (the  number  of  true  positives)  to  all  declared  pixels  (true  positives 
plus  false  positives).  Table  3.2  shows  each  measure  of  algorithm  performance  and 
its  corresponding  response  range. 

3.3  Crossed  Array  Method 

Using  the  controls,  noise,  and  responses  described  above  a  Taguchi-style  crossed 
array  RPD  was  developed.  The  details  for  development  were  adapted  from  the 
Response  Surface  Methodology  text  by  Myers  and  Montgomery  [10]. 

The  design  begins  with  the  inner  array,  and  Tbl.  3.1  suggests  11  control  vari¬ 
ables  to  be  accounted  for  in  the  inner  array.  A  two  level,  full  factorial  experiment  on 
11  variables  would  include  211  =  2048  runs  per  replication.  Additionally,  in  order  to 


3-3 


estimate  quadratic  curvature  of  the  response  across  the  variable  range,  a  three  level 
design  is  desirable  but  increases  the  number  of  runs  required  to  39  x  22  =  78,  732. 
Thus,  some  fractional  factorial  experiment  is  necessary  in  order  to  make  the  ex¬ 
periment  size  practical.  One  such  experiment  type  incorporating  both  three  level 
structure  and  fractional  designs  are  Central  Composite  Designs  (CCD).  CCDs  are 
widely  used  to  fit  second-order  models  because  of  their  high  efficiency  compared  to 
other  quadratic  model  generation  techniques  [8]. 

The  development  of  a  CCD  inner  array  is  complicated  by  the  categorical  nature 
of  two  of  the  eleven  control  variables,  as  shown  in  Table  3.1.  These  two  controls 
must  be  tested  externally  to  the  CCD,  thus  nine  variables  remain.  In  order  to 
avoid  aliasing  of  main  effects  or  two-factor  interactions  with  either  each  other  or 
themselves,  a  resolution  V  CCD  must  be  selected.  The  smallest  design  available  for 
nine  factor  problems  maintaining  the  proper  resolution  contains  64  runs.  This  design 
is  drastically  smaller  than  the  full  factorial  design  leaving  room  for  center  points  to 
be  added  as  well.  Six  center  points  were  added  for  a  total  of  70  runs  within  the 
CCD.  These  center  points  combined  with  the  face  points  generated  by  setting  the 
CCD  alpha  to  one  ensure  sufficient  variation  of  interacting  control  settings  at  any 
one  control’s  observed  levels.  In  order  to  properly  account  for  the  two  level  factors, 
the  CCD  must  be  replicated  at  each  of  the  four  level  combinations  of  the  categorical 
factors  (Threshold  Both  Sides  and  Clean  Signal).  Replicating  the  70  original  CCD 
runs  four  times  yields  280  runs  total  in  the  inner  array. 

In  a  standard  Taguchi-style  experiment,  the  outer  array  is  simply  a  factorial 
(full  or  fractional)  design  based  on  the  noise  variables  under  consideration.  In  this 
case,  only  one  variable  (Noise)  is  being  considered.  This  variable  has  8  discrete 
levels  which  are  representative  of  the  potentially  infinite  real  world  image  library 
that  AutoGAD  may  encounter.  Since  there  is  only  one  factor  at  8  levels,  the  outer 
array  becomes  an  81  full  factorial  design.  This  design  consists  of  8  runs,  one  at  each 
image  level. 
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When  the  the  inner  and  outer  arrays  are  crossed  the  resultant  experiment  is  8 
replications  of  the  inner  array,  one  at  each  noise  level.  The  total  number  of  runs  in 
the  crossed  array  is  280  x  8  =  2240. 

Development  of  the  designed  experiment  was  conducted  using  DesignExpert® 
Version  7.1.5  (DX7).  This  software  package  contains  many  useful  tools  for  designing 
and  analyzing  experiments  of  this  type.  The  desired  experiment  type;  resolution; 
number  of  replications,  blocks  and  center  points;  and  control  factor  and  response 
labels  are  input,  and  one  of  several  outputs  is  a  tillable  spreadsheet  for  use  during 
the  data  collection  portion  of  the  experiment.  The  sheet  generated  for  the  crossed 
array  is  provided  in  a  blockwise  manner  in  Figures  A.l  through  A. 8  in  the  Appendix 
A. 

Now  that  control  and  noise  levels  have  been  developed  for  each  of  the  2240 
runs,  a  means  must  be  devised  to  pass  them  to  AutoGAD.  For  this  purpose  slight 
modifications  were  made  to  the  AutoGAD  function  itself  in  the  way  that  it  reports 
the  responses  of  interest.  Additionally,  a  master  function  was  built  to  iteratively 
load  each  run’s  control  and  noise  levels  and  pass  them  to  AutoGAD  for  execution. 
This  master  function  then  recorded  the  response  values  reported  by  AutoGAD  and 
loaded  up  the  next  runs  values  until  each  run  combination  was  completed  and  it’s 
responses  recorded.  A  copy  of  the  MATLAB®  code  for  AutoGAD  and  the  master 
function  are  included  in  App.  C. 

Upon  completion  of  the  experiment,  the  collected  response  data  along  with  it’s 
corresponding  control  and  noise  parameter  levels  were  exported  to  Microsoft  Excel® 
for  analysis.  While  the  remainder  of  the  analysis  is  straightforward,  it  nonetheless 
requires  several  steps  detailed  below. 

The  first  step  is  to  compute  each  response’s  mean  at  each  design  point  in  the 
inner  array  (IA).  Recall  the  entire  experiment  is  composed  of  8  identical  blocks  which 
vary  only  in  the  noise  factor.  Therefore,  the  inner  array,  consisting  of  280  design 
points  was  repeated  at  each  level  of  the  noise  variable,  and  a  mean  response  at  each 
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of  these  points  can  be  computed  by  summing  the  response  values  observed  at  each 
of  the  noise  levels  and  dividing  by  8.  This  computation  is  expressed  mathematically 
in  Eq.  3.1  where  i/ij  is  the  observed  response  at  design  point  i  and  noise  level  j. 


n 

s  =  £-  (3.1) 

3= i 

The  mean  response  given  by  Eq.  3.1  is  computed  for  each  of  the  four  responses 
at  each  of  the  280  1A  design  points.  Additionally,  SNR’s  are  computed  for  each 
response  at  each  point  on  the  IA.  The  SNR  equation  used  for  this  analysis  is  given 
in  Eq.  2.6.  This  particular  SNR  was  chosen  for  two  reasons.  First,  using  a  SNR  with 
a  target-is-best  approach  allows  it  to  be  applied  to  all  the  responses  of  interest,  even 
though  two  of  the  response  have  target  values  of  zero  while  the  other  two  responses 
have  target  values  of  one.  Second,  as  noted  by  Myers  and  Montgomery  [10],  this  is 
the  only  one  of  Taguchi’s  suggested  SNR  formulations  that  is  truly  unitless. 

The  second  portion  of  the  analysis  is  a  technique  called  marginal  means  model¬ 
ing.  This  analysis  computes  the  marginal  mean  of  the  f/’s  for  each  level  of  a  particular 
control  variable.  This  idea  is  illustrated  in  Tbl.  3.3  where  the  IA  points  are  sorted 
based  on  Dim  Adj.  Then  all  mean  time  responses  for  IA  points  where  Dim  Adj=  —2 
are  averaged  to  find  the  marginal  mean  response  when  Dim  Adj=  —2.  Marginal 
means  are  computed  for  each  level  of  Dim  Adj  observed  in  the  IA.  This  process  is 
repeated  for  each  response  at  each  level  of  each  control  variable  in  the  IA. 

Once  marginal  means  for  y1  s  and  SNR’s  are  computed  at  each  level  of  each 
control  variable,  they  are  then  compared  either  within  the  table  or  by  plotting  the 
values  across  each  control  space  in  a  pick-the- winner  analysis  [10].  The  idea  here 
is  to  pick  the  level  of  each  control  that  has  the  desired  effect  on  either  the  mean 
response,  SNR,  or  both.  Example  plots  generated  using  the  data  in  Tbl.  3.3  are 
shown  in  Fig.  3.1. 
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Control  Variables 

IA  point  response 

Marginal  Means 

Dim 

Adj 

Clean 

Signal 

Mean 

Time 

SNRxime 

Time 

SNR'j'ime 

-2 

0 

29.0887 

-3.0449 

-2 

1 

35.6386 

-4.7822 

32.3637 

-3.9136 

<u 

> 

0 

0 

18.7123 

-5.4859 

0) 

0 

1 

25.1975 

-4.4763 

21.9549 

-4.9811 

2 

1 

17.8785 

-5.6364 

2 

0 

17.4854 

-7.0809 

17.6820 

-6.3587 

Table  3.3  Sample  IA  point  means  and  their  subsequent  Marginal  Means 


Figure  3.1  Sample  Mean  and  SNR  Plots 
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All  that  remains  is  to  select  control  variables  that,  in  our  example,  minimize 
time  while  maximizing  SNR.  Using  the  plots  in  Fig.  3.1,  the  correct  setting  is  to 
let  Dim  Adj  =  0.  This  setting  provides  a  trade  off  between  the  two  competing 
objectives.  While  giving  up  ~3  seconds  from  the  optimal  time,  the  SNR  is  improved 
~1.5.  Maximizing  SNR  only  improves  it’s  solution  by  about  1,  but  has  an  expected 
time  cost  more  than  10  seconds  higher.  Plotting  the  marginal  means  helps  the 
analyst  visualize  the  impacts  of  one  setting  versus  another  and  make  decisions  about 
which  control  variable  settings  have  the  greatest  impact  on  the  response. 

3.4  Combined  Array  Method 

Recall  from  Subsection  2.4.2  that  the  response  surface  approach  has  become 
a  widely  adopted  technique  for  locating  robust  parameter  settings.  Although  RSM 
is  widely  used  for  RPD  problems,  the  literature  is  sparse  with  regard  to  problems 
where  the  noise  parameter(s)  are  categoric  in  nature.  Brenneman  and  Myers  [2] 
published  a  technique  in  2003  that  will  be  adapted  for  use  in  the  AutoGAD  problem 
where  the  noise  variable  is  defined  to  be  the  image  of  interest.  Images  are  discrete 
points  in  the  noise  space  and  therefore  are  easily  translated  in  a  single  categorical 
variable. 

Recall  that  eight  images  are  being  used,  thus  each  can  be  thought  of  as  a 
category  of  a  single  noise  variable,  image.  In  the  real  world  implementation  of  the 
algortihm,  the  probability  of  any  one  image  from  the  infinite  population  of  images 
that  may  exist  being  presented  to  the  algorigthm  is  essentially  zero.  However,  within 
this  experiment,  the  population  is  limited  to  8  and  each  image  has  an  equal  chance 
of  being  presented  to  the  algorithm  at  any  time. 

Just  as  with  the  Crossed  Array  approach,  the  first  step  is  to  develop  a  test 
design.  Development  of  this  design  was  also  conducted  in  DX7.  Initally,  a  full 
factorial  design  was  generated,  consisting  of  ~16000  runs.  These  runs  became  the 
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Input  Parameter 

Classification 

Designator 

Dimension  Adjust 

Control 

A 

Max  Score  Threshold 

Control 

B 

Bin  Width  SNR 

Control 

C 

PT  SNR  Threshold 

Control 

D 

Bin  Width  Indentify 

Control 

E 

Smooth  Iterations  High 

Control 

F 

Smooth  Iterations  Low 

Control 

G 

Low  SNR 

Control 

H 

Window  Size 

Control 

J 

Threshold  Both  Sides 

Control 

K 

Clean  Signal 

Control 

L 

Image 

Noise 

M 

Table  3.4  Control  and  Noise  Factor  Designators 


solution  space  for  DX7’s  D-Optimal  Design  algorithm.  This  algorithm  first  selects  a 
minimum  number  of  runs  needed  to  approximate  a  user-specified  model  complexity. 
As  described  earlier,  a  2-factor  interaction  model  was  specified.  The  algorithm  selects 
the  appropriate  number  of  runs  from  the  solution  space  and  then  by  substitution, 
seeks  to  minimize  the  determinant  of  the  coded  factor  matrix  generated  by  the 
currently  selected  runs.  An  initial  design  of  174  runs  was  suggested  by  the  software, 
however  in  order  to  better  estimate  quadratic  effects,  the  design  was  augmented  with 
32  center-point  runs  for  a  total  of  206  runs. 

The  first  phase  of  analysis  for  the  RSM  approach  is  develop  a  response  model 
using  standard  Analysis  of  Variance  (ANOVA)  techniques.  In  this  case,  ANOVA 
was  accomplished  using  backward  stepwise  regression  on  a  second  order  full  factorial 
model  with  an  a  =  0.1.  Once  again,  the  analysis  process  will  be  illustrated  using  the 
time  response.  For  conciseness  and  clarity  during  analysis,  each  control  and  noise 
variable  was  assigned  a  letter  designator  for  use  in  equations.  Table  3.4  maps  each 
variable  to  it’s  assigned  letter. 

Backward  stepwise  regression  ANOVA  was  performed  on  the  206  time  re¬ 
sponses  and  their  corresponding  control  variables.  A  Box-Cox  test  was  conducted 
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on  the  initial  model.  The  results  of  the  Box-Cox  suggested  the  data  be  transformed 
using  an  inverse  square  transformation.  Eq.  3.2  gives  the  transformation  where  x  is 
the  raw  response  data  and  y  is  the  transformed  response. 


V  =  -2  (3.2) 
xz 

After  transforming  the  data,  ANOVA  was  again  used  to  generate  a  potential 
model.  The  resultant  ANOVA  table  is  shown  in  Table  3.5. 

This  analysis  provided  a  better  fitting  model  than  the  raw  data  ANOVA.  The 
adjusted  R2  for  this  model  was  .78372.  The  equation  for  the  transformed  time  model 
is  given  in  Eq.  3.3. 
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Source 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 

Square 

F 

Value 

p- value 
Prob  <  F 

Model 

3.698975258 

47 

0.078701601 

16.8048762 

<  0.0001 

A 

0.328445766 

1 

0.328445766 

70.13186954 

<  0.0001 

B 

0.007349487 

1 

0.007349487 

1.56931005 

0.2122 

C 

0.001340472 

1 

0.001340472 

0.286226338 

0.5934 

D 

0.000242661 

1 

0.000242661 

0.051814559 

0.8202 

G 

0.000322735 

1 

0.000322735 

0.068912395 

0.7933 

H 

0.024956423 

1 

0.024956423 

5.328857211 

0.0223 

J 

5.36656E-05 

1 

5.36656E-05 

0.011459036 

0.9149 

K 

0.003996599 

1 

0.003996599 

0.853379639 

0.3570 

L 

0.116145608 

1 

0.116145608 

24.80016322 

<  0.0001 

M 

2.362236712 

7 

0.337462387 

72.05715708 

<  0.0001 

AG 

0.023995272 

1 

0.023995272 

5.123626024 

0.0250 

AM 

0.27178284 

7 

0.03882612 

8.290404882 

<  0.0001 

BC 

0.044829954 

1 

0.044829954 

9.57238256 

0.0023 

BK 

0.014034979 

1 

0.014034979 

2.996839694 

0.0854 

BL 

0.073198798 

1 

0.073198798 

15.62988193 

0.0001 

CH 

0.011877226 

1 

0.011877226 

2.53610232 

0.1133 

CK 

0.01527487 

1 

0.01527487 

3.261589385 

0.0728 

DG 

0.026201225 

1 

0.026201225 

5.59465546 

0.0192 

DJ 

0.02874754 

1 

0.02874754 

6.138361227 

0.0143 

GK 

0.01841259 

1 

0.01841259 

3.931575534 

0.0491 

HM 

0.08947022 

7 

0.01278146 

2.729180214 

0.0106 

LM 

0.163885904 

7 

0.023412272 

4.999140129 

<  0.0001 

G2 

0.028935945 

1 

0.028935945 

6.178590651 

0.0140 

Residual 

0.739955049 

158 

0.00468326 

Lack  of  Fit 

0.715930196 

151 

0.00474126 

1.381436854 

0.3481 

Pure  Error 

0.024024853 

7 

0.003432122 

Corr  Total 

4.438930307 

205 

Table  3.5  ANOVA  for  Time  Response 
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Time”2  =  0.095959953  -  0.049083154  x  A  +  0.008216574  x  B 

+  0.004174205  xG  +  0.002180937  x  D  +  0.000517243  x  G  -  0.014463039  x  H 
+  0.000586791  x  J  +  0.005116137  x  K  -  0.024563611  x  L 

-  0.05387493  x  M[  1]  -  0.088532213  x  M[ 2]  +  0.257884684  x  M[ 3] 

+  0.024603022  x  M[ 4]  +  0.004100322  x  M[ 5]  -  0.121417776  x  M[6] 

-  0.068272658  x  M[7]  -  0.013658435  x  AG  +  0.02726298  x  AM[  1] 

+  0.043139191  x  4M[2]  -  0.108957117  x  AM\ 3]  +  0.026570975  x  AM[ 4] 

+  0.000394792  x  4M[5]  +  0.033802758  x  AM[ 6]  -  0.012433922  x  M[7] 

-  0.018644844  x  BC  -  0.010028052  x  BK  +  0.023304115  x  BL 

-  0.010093548  xCH  -  0.010361026  x  CK  +  0.014214098  x  DG 
+  0.01556147  x  DJ  —  0.012026222  x  GK  +  0.011086521  x  HM[  1] 

+  0.017455334  x  HM[ 2]  -  0.062479388  x  MM[3]  +  0.017475338  x  HM[ 4] 

+  0.023545491  x  MM[5]  -  0.01122123  x  MM[6]  +  0.005579995  x  MM[7] 

-  0.007668634  x  LM[  1]  +  0.013823052  x  LM[2]  -  0.008619933  x  LM[3] 

+  0.051810785  x  LM[ 4]  -  0.031730238  x  LM[5]  +  0.019003705  x  LM[6] 

+  0.01267563  x  LM[7]  +  0.03056574  x  G2,  (3.3) 


Notice  that  each  time  the  noise  variable,  M,  occurs  it  is  represented  with  seven 
different  coefficients  and  variable  representations  (M[1]...M[7]).  This  notation  is  a 
result  of  the  dummy  variable  structure  developed  in  Eq.  2.12  where  each  category 
of  the  noise  is  represented  as  a  1  or  0  in  the  equation.  When  the  image  M  equals 
the  first  category,  M[  1]  =  1  and  all  other  M’s  are  zero.  The  eighth  category  is 
the  baseline  where  M[l]  through  M[ 7]  are  all  set  to  zero.  This  structure  allows  the 
model  to  more  accurately  predict  the  response  from  each  category  of  the  noise  space, 
since  those  categories  are  not  continuous  or  even  ordinal  in  nature. 
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3-4-1  Mean  Model 

Now  that  a  model  for  predicting  the  time  response  has  been  developed,  the 
technique  suggested  by  Brenneman  and  Myers  [2]  will  be  adapted  to  create  mean 
response.  Utilizing  the  dummy  variable  structure  developed  in  Eq.  2.12,  along  with 
the  expected  value  for  multinomial  distributions,  E(y)  =  YhiPiVii  the  general  model 
for  estimated  mean  response  across  the  entire  noise  space  is: 


Ez[y(x,  z)]  =  pi£(x,  zi  =  l,z2  =  0,  z3  =  0,z4  =  0,  z5  =  0 ,z6  =  0,  ^7  =  0) 

+  p2y(x,  z  1  =  0,z2  =  1,  ^3  =  0,  z4  =  0,  z5  =  0,z6  =  0,  z7  =  0) 

+  p3y{x,  zi  =  0,z2  =  0,  z3  =  l,z4  =  0,  =  0,  =  0,  z7  =  0) 

+  p4y{x,  z  1  =  0,z2  =  0,  £3  =  0,z4  =  1, =  0,  z6  =  0,  z7  =  0) 

+  P5d(x,  z  1  =  0,z2  =  0,  £3  =  0,  z4  =  0,  z5  =  l,z6  =  0,  z7  =  0) 

+  p6y(x,  z  1  =  0,z2  =  0,  z3  =  0,  z4  =  0,  z5  =  0,  z6  =  1,  z7  =  0) 

+  p7y(x,  z  1  =  0,z2  =  0,  =  0 ,z4  =  0,  z5  =  0 ,z6  =  0,  z7  =  1) 

+  p8y(x,  zt  =0,z2  =  0,  z3  =  0,z4  =  0,  =  0,  =  0,  z7  =  0),  (3.4) 

where  p4 . .  .p8  =  |  for  noise  categories  with  equal  probability  of  occurrence.  How¬ 
ever,  when  developing  a  model  for  the  mean  time  response  from  the  observed  data 
the  probability  of  occurrence  for  each  category  is  easily  calculated  by  dividing  the 
number  of  times  each  image  was  used  by  the  total  number  of  runs.  The  probabilities 
of  occurrence  for  each  category  are  given  in  Table  3.6. 
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Noise 

Category 

Occurence 

Probability 

M[l] 

0.121359223 

M[2] 

0.126213592 

M[3] 

0.13592233 

M[4] 

0.121359223 

M[5] 

0.126213592 

M[6] 

0.116504854 

M[7] 

0.121359223 

M[8] 

0.131067961 

Table  3.6  Observed  Probability  of  Occurrence  for  Noise  Categories 

Plugging  the  probabilities  from  Table  3.6  into  Eq.  3.4  and  combining  the  terms 
results  in  the  reduced  estimated  mean  model  given  in  Eq.  3.5. 


Ez[y(x.,z)\  =  0.094372095 

-  0.049435818  x  A 
+  0.008216574  x  B 
+  0.004174205  x  C 
+  0.002180937  x  D 
+  0.000517243  x  G 

-  0.01494442  x  H 
+  0.000586791  x  J 
+  0.027060476  x  K 
+  0.011679743  x  L 

-  0.013658435  x  AG 

-  0.018644844  x  BC 

-  0.010028052  x  BK 
+  0.023304115  x  BL 

-  0.010093548  x  CH 

-  0.010361026  x  CK 
+301(414214098  x  DG 
+  0.01556147  x  DJ 


-  0.012026222  x  GK 


Since  each  of  the  categorical  noise  variables  in  Eq.  3.3  are  assigned  a  value 
of  1  or  0  during  this  step,  the  noise  factor  interaction  terms  can  be  reduced  to  a 
single  term  where  the  marginal  probability  for  each  noise  category  is  multiplied  by 
the  coefficient  of  it’s  corresponding  control x noise  interaction  term  and  the  resultant 
coefficients  are  summed  along  with  the  coefficient  for  the  control  variable  to  yield  a 
single  coefficient  for  the  control  term.  This  idea  is  illustrated  in  Eq.  3.6  where  T  is 
the  sum  of  all  the  coefficients  and  each  M  is  evaluated  as  1.  Each  of  the  coefficients 
for  the  interaction  terms  results  from  multiplying  the  adjustment  coefficient  for  each 
interaction  term  found  in  Eq.  3.3  by  the  appropriate  occurrence  probability  from 
Table  3.6. 


T  x  A  =  -  0.049083154  x  A 

+  0.003308614  x  AM[  1] 

+  0.005444752  x  AM[ 2] 

-  0.014809705  x  AM[ 3] 

+  0.003224633  x  AM\ 4] 

+  4.98281  *  10”05  x  AM[ 5] 

+  0.003938185  x  AM{ 6] 

-  0.001508971  x  AM[ 7]  (3.6) 


3-4-2  Variance  Model 

Now  that  a  mean  model  has  been  generated,  the  variance  model  follows  directly 
from  Eq.  2.9.  This  version  of  the  variance  model  is  used  since  the  noise  is  categorical 
and  the  p^s  are  known,  so  the  variance-covariance  matrix  is  fully  characterized  and 
its  components  are  easily  substituted  into  equation.  The  estimated  variance  model 
is: 
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Var 2  [t/(x,  z)]  =  ^2  [(t»  +  xA;)2Var(zj) 

i 

+  2E£[  (7 i  +  xA*)(7j  +  xAi)Cov(^,  Zj)  j 

*7 A' 

+  a2  (3.7) 

where  Var(zj)  =  pj(l  —  p.j)  and  Cov(zj,  Zj)  =  —piPj.  Recall  that  7  is  a  vector  of  main 
effect  coefficients  which  in  the  case  of  variance  is  only  the  coefficient  of  the  noise 
main  effect.  Additionally,  A  is  a  matrix  of  control-by-noise  interaction  coefficients. 
Therefore,  the  coefficients  in  Eq.  3.5  provide  all  the  elements  of  both  7  and  A. 
The  remaining  term,  <v2,  is  simply  the  Mean  Square  Error  (MSE)  or  Residual  of  the 
ANOVA  model.  The  coefficients  of  the  variance  model  in  Eq.  3.7  are  thus  computable 
and  the  constant  terms  are  known,  so  all  that  remains  is  to  determine  control  variable 
levels  that  minimize  the  estimated  variance.  Due  to  the  large  number  of  terms  in 
this  model  (over  200),  it  will  not  be  given  here.  However,  the  Excel™  spreadsheet 
used  for  the  step  by  step  analysis  is  included  in  App.  B 

Once  both  the  mean  and  variance  models  are  built,  Brenneman  and  Myers 
[2]  suggest  first  minimizing  the  variance  first  by  adjusting  the  control  factors  that 
interact  with  the  noise.  This  problem  is  notionally  expressed  in  Eq.  3.8. 


min  Var*  [p(x,  z)] 

s.t.  Dispersion  Controls  within  range  of  Designed  Experiment  (3.8) 

The  levels  for  the  dispersion  factors  solved  for  in  Eq.  3.8  are  then  inserted  into 
Eq.  3.5  and  the  remaining  adjustment  control  factors  are  used  to  bring  the  estimated 
mean  response  up,  down,  or  on  target.  All  significant  controls  as  determined  by  the 
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Response 

Transformation 

Adjusted 

R2 

Time 

y  =  Time-2 

0.7837 

True  Positive  Fraction 

y  =  hr  TPF+80 
y  i.ooi-i 

0.9679 

False  Positive  Fraction 

y  =  (FPF  +  0.001)-1 

0.7730 

Target  Fraction  Percent 

y  =  VTFP  +  0.01 

0.6466 

Table  3.7  Transformations  and  Resultant  Adjusted  R2’s 


ANOVA  in  Table  3.5  have  now  been  assigned  values  resulting  in  minimized  response 
variance  and  optimized  response  performance  (min,  max,  target). 

3.5  Reducing  Multiple  Responses  to  a  Single  Dimension 

Recall  from  section  3.2  that  there  are  four  responses  of  interest.  These  re¬ 
sponses  measure  different  aspects  of  AutoGAD  performance  that  can  be  used  in 
aggregate  to  quantify  overall  algorithm  performance.  Initial  analysis  efforts  using 
four  independent  responses  proved  problematic,  especially  within  the  Combined  Ar¬ 
ray  approach. 

First,  the  ANOVA  models  generated  for  each  of  the  responses  fit  the  data 
poorly,  and  as  noted  in  section  3.4,  the  models  failed  Box-Cox  goodness  of  fit  tests. 
DX7  provided  suggested  transformations  along  with  the  results  of  the  Box-Cox  test. 
After  transforming  the  data  for  each  response  according  to  suggestions,  the  resul¬ 
tant  models  were  better,  but  not  sufficient.  Table  3.7  provides  the  transformation 
performed  along  with  the  adjusted  R2’s  for  each  transformed  response’s  model. 

Secondly,  when  the  transformed  response  models  were  used  to  produce  mean 
response  and  expected  variance  models  according  to  technique  described  in  subsec¬ 
tions  3.4.1  and  3.4.2,  these  models  were  poorly  behaved.  The  initial  minimization  of 
the  expected  variance  model  suggested  control  settings  within  the  ranges  observed 
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during  testing,  but  the  generated  responses  often  appeared  improbable,  although 
technically  feasible.  Further,  when  the  dispersion  factor  values  were  inserted  into 
the  mean  response  model,  no  feasible  solution  could  be  found.  Adjustment  factor 
values  located  on  the  ranges  observed  during  the  course  of  the  designed  experiment 
did  not  produce  feasible  response  values.  Thus,  when  adjustment  factor  values  were 
forced  onto  the  observed  range,  the  resultant  mean  response  was  infeasible  (e.g. 
negative  time  values,  TPFS>1). 

Section  3.2  suggests  the  true  measure  of  performance  for  AutoGAD  is  how 
accurately  the  algorithm  identifies  potential  targets  within  a  scene.  Quantifying  this 
performance  resulted  in  the  four  responses  listed  in  Table  3.2.  However,  the  diffi¬ 
culties  detailed  above  precluded  further  analysis  of  individual  responses.  Therefore, 
some  simplification  of  the  responses  was  necessary  in  order  to  complete  the  analysis 
and  locate  robust  parameter  settings. 

The  most  intuitive  simplification  technique  is  to  reduce  the  dimensionality  of 
the  responses.  This  is  most  easily  accomplished  by  developing  some  function  where 
the  responses  listed  in  Table  3.2  are  linearly  combined  in  some  fashion  resulting  in 
a  single  response. 

There  are  two  issues  that  must  be  dealt  with  in  order  to  make  evaluation  of 
this  function  valid.  First,  the  units  of  the  response  data  must  be  either  uniform 
or  removed.  Secondly,  the  scale  of  each  response  must  be  uniform.  The  outlier  in 
both  regards  is  the  time  response.  While  the  three  other  responses  are  essentially 
percentages  and  can  only  exist  on  the  [0, 1]  interval,  the  time  response  is  reported  in 
seconds  and  could  theoretically  be  of  infinite  size. 

A  relatively  simple  and  common  transformation  performed  in  multivariate 
analysis  is  data  standardization  [3].  This  process  begins  by  computing  the  sam¬ 
ple  mean  and  variance  for  each  response.  Next,  the  sample  mean  for  each  respective 
response  is  subtracted  from  each  data  point  in  the  sample.  This  effectively  centers 
each  responses  data  around  its  sample  mean.  The  sample  mean  for  this  adjusted 
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data,  known  as  mean  corrected  data,  is  zero.  Finally,  each  mean  corrected  data  point 
is  divided  by  the  square  root  of  its  respective  sample  variance.  This  step  scales  the 
data  so  that  each  observation  is  now  expressed  in  terms  of  standard  deviations  from 
the  corrected  mean  of  zero.  So,  if  the  data  is  approximately  normally  distributed, 
one  would  expect  that  ~67%  of  the  data  points  would  be  on  the  interval  [—1,1], 
Letting  Xi  represent  the  standardized  form  of  Xi,  the  entire  standardization  process 
is  expressed  in  a  single  step  in  Eq.  3.9. 


x 


i 


(3.9) 


Standardizing  the  data  solves  both  the  unit  and  scale  problems  discussed  above. 
The  standardized  data  is  unitless  since  the  units  in  the  numerator  are  divided  by 
the  square  root  of  squared  units  in  the  denominator.  The  scale  of  standardized 
data  is  now  in  terms  of  standard  deviations  for  all  four  responses.  Each  standard 
deviation  is  computed  using  data  from  a  single  response,  so  it  scales  its  respective 
data  according  to  the  variance  of  the  underlying  distribution  from  which  the  data 
comes. 

The  final  step  before  the  response  terms  can  be  combined  is  to  determine 
the  influence  each  individual  response  will  have  on  the  combined  response.  This 
influence  is  easily  quantified  using  a  weighting  coefficient  for  each  response.  The  size 
of  the  weightings  is  arbitrary,  but  it  is  common  to  select  them  such  that  they  sum 
to  one.  Doing  so  allows  each  weighting  coefficient  to  be  intuitively  translated  into 
a  percentage  that  each  response  will  contribute  to  the  combined  response.  These 
weights  are  commonly  assigned  by  a  process  decision  maker.  For  instance,  in  the 
case  of  AutoGAD,  the  analyst  responsible  for  interpreting  the  output  from  AutoGAD 
would  be  the  decision  maker.  As  such,  they  would  assign  weights  to  each  of  the  four 
responses  according  to  their  individual  preferences  for  speed,  accuracy,  false  alarm 
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rate,  or  some  combination  thereof.  For  the  analysis  here,  each  of  the  four  responses 
will  be  assigned  an  equal  weight  coefficient  of  1.  This  will  give  equal  importance  to 
each  in  determining  the  combined  response  value  from  a  particular  combination  of 
parameter  settings. 

Generating  the  objective  function  is  now  a  straightforward  matter,  however 
care  must  be  taken  to  ensure  each  response  contributes  to  the  function  in  the  proper 
direction.  Considering  each  of  the  four  responses  independently,  the  sign  their  re¬ 
spective  terms  carry  into  the  objective  function  is  determined.  Before  this  can  begin 
the  objective  function  type  is  determined  and  subsequently  determines  the  sign  for 
each  of  its  contributing  responses.  Maximizing  the  objective  function  is  intuitive  for 
optimizing  a  target  detection  algorithm  and  thus  will  be  used  here. 

The  first  response  under  consideration  is  the  time  response  which  is  optimized 
by  minimizing  the  run  time  of  the  algorithm  and  delivering  solutions  to  the  user  as 
quickly  as  possible.  Therefore,  the  time  term  in  the  objective  function  should  have 
a  negative  sign  so  that  as  the  time  response  grows  smaller  the  objective  function 
experiences  smaller  penalties  from  the  observed  time  response.  Recall  the  time 
responses  have  been  standardized,  so  the  smallest  times  will  enter  the  objective 
function  as  negative  values  since  they  are  smaller  than  the  mean  which  was  shifted  to 
zero.  Subtracting  this  already  negative  value  effectively  adds  it  thereby  maximizing 
the  resultant  objective  function  value. 

Next  is  the  true  positive  fraction  response.  Since  this  response  is  optimized 
at  its  maximum  value  of  one,  adding  its  term  within  the  objective  function  further 
increases  the  objective  function’s  value.  Standardizing  the  TPF  data  has  a  similar 
but  opposite  effect  to  what  was  observed  in  the  time  response.  Since  the  largest  TPF 
values  are  most  desirable,  they  are  larger  than  the  standardized  mean  response  of 
zero  and  will  therefore  be  positive.  Adding  this  positive  value  has  the  desired  effect 
on  the  objective  function  value. 
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The  third  response  considered  is  false  positive  fraction.  This  response  is  opti¬ 
mized  at  its  minimum  value  of  zero.  Thus,  its  standardized  term  within  the  objective 
function  is  treated  in  the  same  manner  as  the  time  response  term.  Its  optimal  stan¬ 
dardized  value  is  below  the  mean  and  therefore  negative.  Once  again,  by  subtracting 
this  negative  value  it  is  effectively  added  and  the  objective  function  value  increases 
accordingly. 

The  final  response  considered  is  target  fraction  percent.  This  response  behaves 
like  TPF  and  is  optimized  at  its  maximum  value  of  one.  The  optimal  standardized 
TFP  response  is  greater  than  its  standardized  mean  and  thus  positive.  Therefore,  like 
TPF  its  term  is  added  within  the  objective  function  and  serves  to  further  maximize 
the  objective  function  value. 

All  four  responses  are  now  sufficiently  prepared  such  that  they  can  be  combined 
into  a  single  objective  function.  This  function  is  mathematically  expressed  in  general 
form  in  Eq.  3.10  where  O  represents  the  objective  function’s  response  value  and  x 
represents  the  standardized  individual  responses.  Maximizing  this  one  dimensional 
response  will  drive  the  individual  responses  toward  their  respective  optimality  values. 


max  O  =  — 


1  x  Time 


+ 


1  x  TPF 


1  x  FPF 


+ 


1  x  TFP 


(3.10) 


Creating  this  new  one  dimensional  response  for  each  point  in  the  design  space 
generates  a  new  data  set  that  can  be  analyzed  in  exactly  the  same  manner  as  the 
original  responses.  Within  the  Combined  Array  approach  detailed  in  Section  3.4, 
ANOVA  can  be  performed  to  generate  a  predictive  model.  Analysis  of  the  resul¬ 
tant  model  to  generate  models  for  expected  mean  response  and  expected  variance 
is  performed  in  exactly  the  same  manner.  The  subsequent  determination  of  control 
variable  settings  is  also  accomplished  according  to  the  methods  previously  discussed. 
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While  the  Crossed  Array  approach  did  not  necessarily  exhibit  the  same  severe 
pathology  during  initial  analysis  of  the  individual  responses,  similar  dimensionality 
reduction  was  conducted  for  the  sake  of  consistency.  Recall  from  Section  3.3,  y’s 
and  SNR’s  were  computed  for  each  response  at  each  of  the  280  design  points  in  the 
IA.  The  2240  responses  were  standardized  using  Eq.  3.9  prior  to  computing  y’s  and 
SNR’s  for  each  of  the  280  IA  design  points.  The  objective  function  for  combining  the 
y’s  accordingly  uses  the  procedure  detailed  above  for  determining  the  signs  of  each 
response’s  term  within  the  objective  function.  So,  the  standardized  single  dimension 
objective  value  is  computed  using  Eq.  3.11. 


max  y  —  —  yTime  +  2/tpf  —  Vfpf  +  Vtfp  (3.11) 

SNR’s  are  already  unitless  and  scaled  by  the  sample  variance  according  to 
Eq.  2.6,  thus  they  do  not  require  standardization.  Additionally,  regardless  of  the 
optimal  value  of  the  response  that  produced  a  particular  SNR,  it  is  optimized  by 
maximization.  Thus,  the  objective  function  for  the  combined  SNR’s  is  simply  a  sum 
as  shown  in  Eq.  3.12. 


max  SNRTotal  =  SNRTime  +  SNRTPF  +  SNRFPF  +  SNRTFP  (3.12) 
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4.  Analysis  of  Results 

This  chapter  provides  detailed  analysis  of  experimental  results  from  two  separate 
RPD  techniques.  The  analysis  methodology  employed  within  each  technique  is  de¬ 
tailed  in  Chapter  3. 

All  experimental  AutoGAD  runs  for  both  methodologies  were  conducted  on  a 
Dell  Precision™  490  PC  with  a  3.0  GHz  Intel®  Xeon®  5160  with  3.0  GB  of  RAM 
running  Windows™  XP  SP3.  The  performance  data  was  collected  and  recorded 
during  the  run  execution,  but  the  computation  time  for  data  collection  did  not 
contribute  to  the  per  run  algorithm  performance  time. 

4-1  Analysis  of  the  Crossed  Array 

Within  this  section  the  results  of  the  Crossed  Array  analysis  are  presented 
in  keeping  with  the  steps  of  the  analysis  procedure  detailed  in  Section  3.3.  Recall 
from  Table  3.1  there  are  11  control  variables  and  the  inner  array  is  a  70  run  CCDy 
replicated  four  times  yielding  280  design  points  on  the  inner  array.  The  control 
variable  levels  for  each  design  point  can  be  found  along  with  their  corresponding  raw 
responses  in  Tables  A.l  through  A. 8  in  App.  A. 

The  rows  in  each  of  the  eight  tables  correspond  to  each  other,  meaning  the 
control  variable  levels  in  row  one  of  A.l  are  exactly  the  same  as  the  levels  found 
in  row  one  of  the  other  seven  tables.  This  set  of  rows  make  up  the  design  points 
in  the  outer  array  of  the  experimental  design.  The  data  in  all  of  the  tables  are 
first  standardized  with  respect  to  the  set  of  all  2240  points.  These  standardized 
responses  across  the  eight  matching  rows  of  the  tables  can  be  averaged  to  find  a 
mean  standardized  response  across  the  noise  space.  Doing  so  for  each  of  the  2240 
data  points  captured  by  the  tables  referenced  above  reduces  the  data  to  280  points, 
one  mean  response  for  each  inner  array  design  point.  The  overall  mean  and  variance 
of  these  280  mean  responses  is  now  easily  computed  and  used  to  produce  a  SNR  for 
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each  point.  The  SNR’s  were  computed  using  Eq.  2.6  and  are  provided  along  with 
the  mean  responses  for  each  of  the  4  algorithm  outputs  in  Table  A. 9  in  App.  A. 

The  data  for  mean  response  is  now  a  set  of  280,  four  dimensional  vectors 
where  each  dimension  represents  one  of  the  four  responses  of  interest.  At  this  point, 
the  dimensionality  reduction  process  detailed  in  section  3.5  was  used  to  reduce  the 
four  dimensional  response  data  to  a  single  response  according  to  Eq.  3.11.  The 
individual  response  SNR’s  were  also  combined  using  Eq.  3.12.  Both  of  these  new 
one  dimensional  data  sets  are  provide  in  Table  4.1. 

The  next  step  in  the  crossed  array  analysis  is  to  compute  the  marginal  means 
for  each  of  the  levels  of  the  control  variables.  This  is  easily  done  by  concatenating  the 
data  in  Table  4.1  with  the  levels  of  the  control  variable  of  interest  and  then  resorting 
the  new  table  by  the  control  variable  column.  Now  the  response  data  is  grouped  by 
control  variable  level  and  the  mean  of  each  group  is  easily  computed.  Repeatedly 
resorting  the  data  using  a  new  control  variable  for  each  iteration  produces  marginal 
mean  responses  for  each  level  of  the  control  variables.  These  marginal  means  are 
shown  in  Table  4.2. 

The  marginal  means  for  each  control  variable  can  now  be  plotted,  vizualizing 
the  relationship  between  the  response  and  the  control  of  interest.  This  visualization 
communicates  more  clearly  the  control  x  response  relationship  and  assists  in  the  pick- 
the-winner  analysis.  The  marginal  means  of  the  SNR’s  are  also  plotted  to  assist  in 
selecting  control  variable  settings  that  maximize  SNR,  thereby  minimizing  response 
variance.  The  mean  response  and  SNR  plots  for  each  control  variables  are  given  in 
Figures  4.1  through  4.11. 

Now  that  the  marginal  means  for  mean  response  have  been  computed  and 
plotted,  decisions  must  be  made  about  which  control  variables  will  be  used  to  bring 
the  mean  responses  to  target  (maximize  the  objective  function  of  standardized  re¬ 
sponses)  and  which  will  be  used  to  maximize  SNR,  or  minimize  response  variance. 
Even  though  Myers  and  Montgomery  suggest  simultaneous  mean  response  and  SNR 
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Run 

Summed  Std  Means 

Summed  SNRs 

Run 

Summed  Std  Means 

Summed  SNRs 

Run 

Summed  Std  Means 

Summed  SNRs 

Run 

Summed  Std  Means 

Summed  SNRs 

1 

0.167490866 

-68.68018497 

71 

0.359452577 

-81.88113504 

141 

-0.385459391 

-46.97569487 

211 

-1.532958652 

-36.96910486 

2 

0.43247392 

-50.86710022 

72 

-1.661642317 

-61.44823209 

142 

0.209096854 

-75.97631302 

212 

-0.102495347 

-76.04069189 

3 

-1.475964427 

-59.98940148 

73 

0.32693422 

-52.08113201 

143 

-0.302678613 

-17.18930107 

213 

1.340071013 

-11.01833206 

4 

-0.423155471 

-84.22641974 

74 

0.0427316 

-56.61957211 

144 

-0.152829872 

-45.37047022 

214 

-0.153078555 

-24.93500636 

5 

-0.865821793 

-10.36401404 

75 

-1.002863973 

-14.01865182 

145 

0.431961998 

-26.16829706 

215 

0.520655319 

-63.02692864 

6 

0.126705182 

-3.223114945 

76 

0.438638627 

-33.95417154 

146 

1.299145213 

-17.18573367 

216 

-1.687966101 

-46.59987214 

7 

-1.272567142 

-13.43290864 

77 

0.713783529 

-65.43198514 

147 

0.43316236 

-42.43600742 

217 

-1.000577247 

-16.22445876 

8 

-0.714242327 

-98.7758339 

78 

-0.244732452 

-69.25729349 

148 

-0.61081448 

-32.73280375 

218 

-0.375403449 

-39.26444577 

9 

-1.291217747 

-58.10549898 

79 

-0.624597314 

-96.78771938 

149 

1.261690126 

-24.91917975 

219 

-0.341098518 

-40.28433249 

10 

-1.274876887 

-54.57572264 

80 

-1.056185006 

-17.07356802 

150 

1.419376282 

-13.25579461 

220 

-0.223952141 

-37.5735844 

11 

-1.006905912 

-24.02089928 

81 

0.32380038 

-60.94340843 

151 

0.304531447 

-77.78593462 

221 

-0.558900537 

-56.2106208 

12 

-0.851549321 

-56.75485408 

82 

-0.185354477 

-54.34484069 

152 

-0.10540833 

-95.60106157 

222 

-0.557999836 

-39.12564767 

13 

-0.912438743 

-41.09795254 

83 

-0.098622674 

-70.25234981 

153 

1.183658451 

-13.54710196 

223 

0.253377141 

-17.97947887 

14 

0.449198801 

-51.46450153 

84 

-2.228668894 

-32.24705654 

154 

-1.057064769 

-58.75612591 

224 

-0.114738627 

-8.82892657 

15 

-2.175993877 

2.944451956 

85 

0.340187769 

-48.6304648 

155 

-0.976049812 

-56.81684434 

225 

0.956554974 

-40.9361939 

16 

-1.095284684 

-39.29791692 

86 

-0.384062466 

-37.99870654 

156 

-0.369337933 

-28.83477028 

226 

-0.670952719 

-90.37779174 

17 

1.240612211 

-29.53434677 

87 

0.420146843 

-36.1979395 

157 

1.264568368 

-14.85721767 

227 

-0.035507283 

-82.40650367 

18 

1.350029394 

-10.05942784 

88 

0.538258943 

-59.335434 

158 

0.665277985 

-13.63596233 

228 

0.399853396 

-74.10523174 

19 

0.733173894 

-8.096842112 

89 

-1.142762916 

-82.94635351 

159 

1.081265636 

-53.13627276 

229 

1.070885976 

-22.37624356 

20 

-0.571347987 

-84.78819564 

90 

-0.84115702 

-52.80578911 

160 

-1.03422301 

-82.94634892 

230 

0.697256224 

-17.1412427 

21 

-0.332759344 

22.32023837 

91 

1.340027011 

0.287876096 

161 

1.615038987 

3.506423736 

231 

0.466620308 

-40.55979132 

22 

-0.081374223 

-90.76581322 

92 

-0.136808333 

-43.94703371 

162 

-0.552805961 

-84.29619592 

232 

0.640615481 

-30.57712292 

23 

-1.607715837 

-51.14304535 

93 

1.021360223 

-39.89002439 

163 

-0.715617092 

-60.38231894 

233 

0.329359401 

-80.85220783 

24 

-1.022453579 

-47.60222078 

94 

-1.098908671 

-62.4194094 

164 

0.16830093 

-44.3861619 

234 

0.706177565 

-41.10965615 

25 

-0.387085691 

-42.8104341 

95 

1.036657717 

-2.625243637 

165 

-0.900101746 

-47.32202306 

235 

0.093214258 

-80.53802792 

26 

-0.384777651 

-63.3248773 

96 

0.154507146 

-58.50844089 

166 

0.766711878 

-36.31927661 

236 

1.051381382 

-7.997924567 

27 

1.379649609 

-22.134268 

97 

-0.454408826 

-70.91850658 

167 

1.642870623 

19.97218516 

237 

0.320006671 

-64.23641949 

28 

0.440512198 

-44.36181812 

98 

1.274210429 

-8.908705297 

168 

1.203984752 

-12.27822147 

238 

-0.75372455 

-31.33641369 

29 

0.132054507 

-52.25641604 

99 

0.767088053 

-66.45824768 

169 

1.138335886 

-20.7675684 

239 

0.062513719 

-2.450860459 

30 

-0.944833397 

-85.08825777 

100 

-1.207177063 

-46.30125196 

170 

1.787933024 

27.08410726 

240 

1.179932221 

-32.12133117 

31 

1.581393501 

0.246975404 

101 

0.5195634 

-34.00998834 

171 

-0.047131523 

-84.798662 

241 

0.114156052 

-29.05245416 

32 

0.399114324 

-56.72203174 

102 

0.280995113 

-111.4060113 

172 

-0.016963111 

-66.53949982 

242 

-0.315280215 

-68.58478092 

33 

-1.049830944 

-68.05572052 

103 

-0.451486239 

-58.4529208 

173 

0.365169056 

-44.90442186 

243 

0.371308299 

-51.22554132 

34 

-0.424298216 

2.438586625 

104 

-0.864721568 

-51.37522778 

174 

-1.28313677 

-34.85911905 

244 

-1.565265524 

-35.18902039 

35 

0.948007309 

-33.96070283 

105 

-0.074898286 

-25.37320739 

175 

0.164930804 

-48.23276754 

245 

0.627729123 

-39.40945214 

36 

0.528468709 

-39.58130512 

106 

-0.930433857 

-70.88099462 

176 

-1.469773527 

-70.02972027 

246 

0.451675349 

-57.09866882 

37 

1.55344953 

-13.46801799 

107 

-0.120175751 

-94.79505576 

177 

-0.529098278 

-74.52789903 

247 

-0.619416669 

-32.34973289 

38 

1.422072939 

2.472434771 

108 

-1.762078101 

-41.81118407 

178 

-0.034626245 

-70.15889954 

248 

0.654098034 

-41.68003445 

39 

0.331647924 

-40.76856952 

109 

-1.135759779 

-26.99771377 

179 

0.064839957 

-68.26227597 

249 

0.362423439 

-59.89379431 

40 

-0.991975371 

-38.30978014 

110 

0.872473861 

-60.0306535 

180 

0.541325356 

-39.81398473 

250 

0.693359928 

-30.71830906 

41 

0.296052248 

-19.14627811 

111 

-0.776548999 

-22.44685293 

181 

0.588986194 

-46.41936011 

251 

1.74111942 

-4.267731525 

42 

-1.191990612 

-50.25199615 

112 

-0.602451391 

-78.87270543 

182 

-0.294577111 

-122.966282 

252 

-0.17354619 

-54.41110569 

43 

-1.153923091 

-0.857682591 

113 

-0.012898977 

-73.79352424 

183 

-0.008812783 

-38.77674112 

253 

0.779343454 

-71.72336868 

44 

-0.228336808 

-76.30220916 

114 

0.546111532 

-26.49649566 

184 

0.820277969 

-50.41663634 

254 

1.511120617 

23.73359879 

45 

-0.590321371 

-15.71126846 

115 

1.233242034 

-10.48645859 

185 

-1.095405636 

-43.11666374 

255 

-0.640618222 

-16.63582598 

46 

0.261285707 

-48.9797643 

116 

1.064664514 

-23.00064896 

186 

0.648967307 

-35.10581099 

256 

0.609708405 

-63.75118754 

47 

0.084302351 

-57.40017881 

117 

-0.311507549 

-38.26447445 

187 

0.266851545 

-17.07180848 

257 

-1.507666632 

-50.33349786 

48 

-1.997048166 

-42.1084305 

118 

0.483964964 

-15.91170162 

188 

0.424617704 

-85.9166712 

258 

-0.599809085 

-34.21638406 

49 

1.924805093 

16.62415719 

119 

0.413390187 

-62.30630167 

189 

-0.185649041 

-31.31362475 

259 

0.558468752 

-19.79668572 

50 

0.157616228 

-66.43637686 

120 

0.002460612 

-63.28521761 

190 

-0.837849279 

-37.9526951 

260 

0.292026698 

-37.88334964 

51 

-0.06074629 

-37.09936242 

121 

1.261322503 

-42.40153026 

191 

-0.185572231 

-22.27012747 

261 

0.824135651 

-32.94496958 

52 

0.273063125 

-53.07052003 

122 

-0.738349171 

-35.40871188 

192 

0.288595622 

-54.64127892 

262 

-1.530472545 

-38.20670788 

53 

-0.50852189 

-76.24068583 

123 

-1.354935639 

-67.08000044 

193 

-2.048558878 

-56.16907924 

263 

1.196301233 

14.21760949 

54 

-0.504834952 

-24.95073459 

124 

-0.634354533 

-70.80860799 

194 

0.142129129 

-33.16338036 

264 

1.83386557 

33.56463273 

55 

-0.350125921 

-64.57741508 

125 

0.442051457 

-39.57496328 

195 

1.439595698 

-15.49133982 

265 

-0.328005752 

-83.50305901 

56 

-0.095054906 

-52.94813533 

126 

-0.662946103 

-77.02233759 

196 

-1.389423248 

-29.17588017 

266 

-0.173303714 

-110.2637689 

57 

0.68039304 

-9.318750123 

127 

0.424369327 

-71.91469678 

197 

-0.536223597 

-81.76708224 

267 

0.799435094 

-5.167729317 

58 

-1.758333257 

-28.16890731 

128 

-0.241906304 

-60.2050598 

198 

0.758260591 

-21.26987935 

268 

-0.077732983 

-66.82067371 

59 

-0.437141551 

-86.92790743 

129 

1.558318477 

4.440382217 

199 

-1.368463836 

-65.67214658 

269 

-1.733105132 

-7.524267988 

60 

-0.756971219 

-24.93994675 

130 

-0.303691598 

-66.78725398 

200 

-0.445590175 

-77.3894462 

270 

-1.179909403 

-63.11897869 

61 

0.102161657 

-52.08996107 

131 

-0.086777646 

-61.47511333 

201 

1.522685516 

1.966589232 

271 

-0.003167298 

-97.1230054 

62 

1.644279586 

20.61176811 

132 

-0.858067555 

-39.54561608 

202 

1.026266858 

-32.56352378 

272 

0.623707695 

-24.38408899 

63 

0.980932764 

-38.57692714 

133 

-0.855351737 

-45.49482725 

203 

-0.724267635 

-64.19372213 

273 

-1.295357602 

-57.79203774 

64 

-0.428001985 

-57.98841084 

134 

0.586952864 

-72.090623 

204 

-0.176009404 

-27.31353232 

274 

0.717202452 

-6.031695521 

65 

0.334685029 

-53.02719121 

135 

0.148901256 

-67.81409502 

205 

-1.451031745 

-42.87771641 

275 

-0.726532496 

-32.31194 

66 

-0.108400001 

-67.6993738 

136 

-0.051319188 

-67.21593806 

206 

1.545815918 

9.286474723 

276 

1.028656355 

-56.3533817 

67 

1.682518632 

5.767818855 

137 

0.174526236 

-54.92455577 

207 

-1.687643208 

-38.50560214 

277 

-0.031963248 

-16.00370282 

68 

0.48946526 

-93.64314742 

138 

0.731909417 

-36.46113246 

208 

1.364430122 

2.880584423 

278 

-0.319270023 

-67.39422981 

69 

0.302990927 

-44.85683084 

139 

0.163945327 

-35.53210086 

209 

0.603135179 

-49.79559266 

279 

1.084173269 

1.179483512 

70 

1.535821388 

-3.035088258 

140 

-0.168564946 

-5.534704469 

210 

-1.195273831 

-25.48655465 

280 

0.408991039 

-41.70344649 

Table  4.1 


One  Dimensional  Standardized  Mean  Responses  and  SNR’s 
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(b)  Dimension  Adjust  vs.  SNR 

Figure  4.1  Dimension  Adjust  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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(a)  Max  Score  vs.  Mean  Response 
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(b)  Max  Score  vs.  SNR 

Figure  4.2  Max  Score  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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Bin  Width  SNR 
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(b)  Bin  Width  SNR  vs.  SNR 

Figure  4.3  Bin  Width  SNR  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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PT  SNR  thresh 
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(a)  PT  SNR  Threshold  vs.  Mean  Response 
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(b)  PT  SNR  Threshold  vs.  SNR 

Figure  4.4  PT  SNR  Threshold  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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(a)  Bin  Width  Ident  vs.  Mean  Response 
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(b)  Bin  Width  Ident  vs.  SNR 

Figure  4.5  Bin  Width  Ident  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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Smooth  Iter  High 
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(b)  Smooth  Iter  High  vs.  SNR 

Figure  4.6  Smooth  Iter  High  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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Smooth  Iter  Low 
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(b)  Smooth  Iter  Low  vs.  SNR 

Figure  4.7  Smooth  Iter  Low  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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(b)  Low  SNR  vs.  SNR 

Figure  4.8  Low  SNR  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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(a)  Window  Size  vs.  Mean  Response 
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(b)  Window  Size  vs.  SNR 

Figure  4.9  Window  Size  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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Threshold  Both  Sides 
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(a)  Threshold  Both  Sides  vs.  Mean  Response 
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(b)  Threshold  Both  Sides  vs.  SNR 

Figure  4.10  Threshold  Both  Sides  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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(b)  Clean  Signal  vs.  SNR 

Figure  4.11  Clean  Signal  vs.  Marginal  Means  of  Mean  Response  and  SNR 
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Control 

Variable 

Selected 

Level 

Expected  Standardized 
Mean  Response 

Dimension  Adjust 

-2 

0.084166467 

Max  Score 

6 

0.05696234 

Bin  Width  Indent ify 

0.1 

0.079172116 

Smooth  Iterations  High 

50 

0.05862537 

Smooth  Iteration  Low 

25 

0.056707338 

Low  SNR 

4 

0.045094616 

Clean  Signal 

1 

0.090871599 

Table  4.3  Significant  Controls  Variables,  Their  Levels,  and  Expected  Standardized 
Mean  Responses 

analysis  to  determine  control  variable  levels  [10],  after  close  examination  of  the  data 
in  Table  4.2  and  Figures  4.1  through  4.11,  levels  were  first  selected  to  adjust  mean 
response  appropriately,  while  the  remaining  control  variables  were  used  to  maximize 
SNR, 


Regardless  of  selection  methodology,  criteria  must  be  developed  to  determine 
which  control  variables  exert  large  enough  influence  on  either  mean  response  or  SNR 
to  be  categorized  as  significant  to  one  or  both  measures.  Only  one  heuristic  required 
development  during  analysis,  since  mean  response  was  maximized  prior  to  consider¬ 
ing  SNR,  The  heuristic  developed  for  determining  which  control  variables  were  sig¬ 
nificant  contributors  in  maximizing  mean  response  was  simple  and  straightforward. 
If  any  observed  level  for  a  particular  control  produced  a  standardized  marginal  mean 
response  >  0.04,  then  the  control  was  deemed  significant  to  mean  response  and  the 
level  producing  the  max  response  selected  as  robust  in  the  design  space.  Applying 
this  heuristic  to  the  marginal  mean  responses  in  Tabic  4.2  resulted  in  selecting  seven 
of  11  control  variables  as  significant  to  mean  response.  These  controls,  along  with 
their  selected  levels  and  the  expected  mean  response  at  these  levels  are  presented  in 
Table  4.3. 

Selecting  mean  response  controls  is  the  first  half  of  the  selection  process.  Anal¬ 
ysis  of  the  remaining  controls  must  now  be  done  to  determine  the  appropriate  levels 
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to  maximize  the  SNR  and  thus  minimize  response  variance.  This  portion  of  the  anal¬ 
ysis  is  truly  a  pick-the-winner  exercise  and  the  plots  of  control  vs.  SNR  in  Figures 
4.1  through  4.11  make  the  analysis  elementary.  Selecting  the  point  in  each  of  the 
appropriate  plots  and  noting  it’s  corresponding  level  resulted  in  the  expected  SNR’s 
presented  in  Table  4.4. 


Control 

Variable 

Selected 

Level 

Expected 

SNR 

Bin  Width  SNR 

0.06 

-41.64158433 

PT  SNR  Threshold 

3.5 

-42.04400436 

Window  Size 

5 

-41.03432788 

Threshold  Both  Sides 

1 

-40.99105378 

Table  4.4  Significant  Controls  Variables,  Their  Levels,  and  Expected  SNR 


4-2  Combined  Array 

Analysis  of  the  results  of  the  combined  array  RPD  problem  for  AutoGAD 
will  be  provided  in  this  section.  Recall  from  section  3.4  the  technique  utilized  is 
adapted  from  a  methodology  presented  by  Brenneman  and  Myers  [2]  and  is  unique 
in  that  it  treats  noise  as  a  categorical  variable.  Development  of  the  experimental 
design  is  also  detailed  in  section  3.4.  The  end  result  of  the  experimental  design  was 
a  174-run  D- Optimal  design  augmented  with  32  center  point  runs  for  a  total  of  206 
runs. 

The  runs  were  conducted  assigning  control  variable  levels  according  to  the 
randomized  experimental  design.  Both  control  levels  and  response  results  for  each 
of  the  206  runs  is  presented  in  Table  B.l  in  App.  B. 

As  discussed  at  length  in  section  3.5,  analysis  of  the  individual  response  param¬ 
eters  was  problematic  and  a  dimension  reduction  strategy  was  therefore  employed. 
In  a  fashion  similar  to  the  response  transformations  for  the  crossed  array  analysis, 
the  combined  array  data  was  first  standardized  to  deal  with  response  unit  and  scal- 
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ing  issues.  This  standardized  data  reported  in  Table  4.5  was  computed  using  Eq. 
3.9. 

These  standardized  response  data  were  then  reduced  in  dimensionality  ac¬ 
cording  to  Eq.  3.10  to  develop  a  one  dimensional  response.  The  one  dimensional 
standardized  response  data  used  in  completion  of  the  combined  array  analysis  are 
provided  in  Table  4.6. 

The  first  step  in  the  remaining  analysis  is  to  perform  ANOVA  on  the  data  in 
Table  4.6.  As  detailed  in  Section  3.4,  ANOVA  was  accomplished  using  a  backward 
stepwise  regression  with  an  a  =  0.1.  The  ANOVA  resulted  in  a  quadratic  model 
with  17  terms  and  an  adjusted  R2  of  0.663196.  The  ANOVA  table  is  shown  in  Table 
4.7. 

While  the  adj-R2  for  this  model  is  lower  than  three  of  the  four  from  the  in¬ 
dividual  responses,  recall  both  the  raw  and  transformed  response  models  produced 
infeasible  predictions.  The  equation  for  the  one  dimensional  standardized  response 
model  is  given  in  Eq.  4.1. 
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Table  4.5  Combined  Array  Standardized  Response  Data 
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Std  Run 

Std  Total 

Std  Run 

Std  Total 

Std  Run 

Std  Total 

Std  Run 

Std  Total 

Std  Run 

Std  Total 

1 

-0.202785 

43 

1.8224309 

84 

2.9385544 

125 

1.3636977 

166 

-2.571426 

2 

-2.092313 

44 

-1.587695 

85 

-2.802948 

126 

1.3070039 

167 

-2.243681 

3 

2.9388727 

45 

0.5575618 

86 

0.4452896 

127 

2.8966362 

168 

2.3734457 

4 

-2.120113 

46 

-2.258423 

87 

-2.060523 

128 

1.5273413 

169 

2.3586699 

5 

2.9639548 

47 

-1.260269 

88 

-2.111192 

129 

2.9648454 

170 

2.5539239 

6 

0.0077118 

48 

-2.755418 

89 

-6.864299 

130 

2.9338424 

171 

2.5087822 

7 

1.6253994 

49 

2.6060346 

90 

-0.553314 

131 

2.5310482 

172 

0.7416251 

8 

1.4231291 

50 

2.9613986 

91 

0.8828077 

132 

-4.260626 

173 

0.7750064 

9 

0.4037519 

51 

2.9265125 

92 

-2.326091 

133 

-2.391722 

174 

-0.643483 

10 

-2.210422 

52 

0.7048222 

93 

-2.779955 

134 

-3.546562 

175 

-0.405804 

11 

0.4916339 

53 

-3.249205 

94 

-1.707072 

135 

2.2595772 

176 

2.9333317 

12 

1.1454256 

54 

-2.451591 

95 

2.9287248 

136 

-2.522298 

177 

2.9596098 

13 

-2.521256 

55 

-2.260797 

96 

0.9500219 

137 

-3.019912 

178 

2.9549516 

14 

0.8332834 

56 

0.8051304 

97 

-2.122046 

138 

2.9606672 

179 

2.9534194 

15 

0.2085339 

57 

1.4073386 

98 

1.3031638 

139 

-4.946614 

180 

-0.687862 

16 

1.9349669 

58 

-2.07675 

99 

-1.851682 

140 

2.5248854 

181 

0.1959976 

17 

-1.746312 

59 

2.94099 

100 

-0.936765 

141 

-0.612504 

182 

0.2315297 

18 

-2.396992 

60 

2.928848 

101 

2.9194547 

142 

-0.749627 

183 

-0.603135 

19 

-2.177969 

61 

-0.537195 

102 

2.9490462 

143 

0.1566107 

184 

2.5692265 

20 

-2.055944 

62 

-0.872442 

103 

-2.391513 

144 

-2.257789 

185 

2.5913382 

21 

-5.298087 

63 

1.4046115 

104 

-5.798018 

145 

-2.22113 

186 

2.4917643 

22 

-2.202694 

64 

-3.041743 

105 

-1.755848 

146 

-2.173534 

187 

2.4911646 

23 

2.9413467 

65 

-2.264329 

106 

-2.40241 

147 

2.9601487 

188 

-2.821767 

24 

0.421769 

66 

-2.353729 

107 

1.4335063 

148 

2.8898838 

189 

-1.187776 

25 

2.5416055 

67 

-2.826959 

108 

1.22675 

149 

-2.58154 

190 

-0.870474 

26 

-3.702155 

68 

2.9468801 

109 

2.5432126 

150 

-0.938309 

191 

-0.462659 

27 

0.7137967 

69 

2.3641341 

110 

-2.519978 

151 

1.4644465 

192 

0.962214 

28 

2.9357852 

70 

1.8590369 

111 

2.9467851 

152 

1.0312045 

193 

1.6426322 

29 

1.3136734 

71 

-4.001154 

112 

2.9325617 

153 

1.0881592 

194 

1.5050428 

30 

-2.083913 

72 

1.902546 

113 

2.5990873 

154 

-2.146841 

195 

2.1480062 

31 

-2.185037 

73 

1.5552272 

114 

-6.152102 

155 

0.4615024 

196 

-0.241729 

32 

2.337181 

74 

1.2235748 

115 

-3.79168 

156 

2.9370068 

197 

1.3808834 

33 

2.9506168 

75 

1.0247453 

116 

1.4155805 

157 

2.4905042 

198 

0.3856018 

34 

1.7409024 

76 

0.4927919 

117 

1.5124396 

158 

-0.193761 

199 

0.4571809 

35 

-2.060247 

77 

2.9559731 

118 

1.0264573 

159 

1.7765763 

200 

-2.218088 

36 

-2.086077 

78 

-0.853334 

119 

-2.146971 

160 

-2.443783 

201 

-4.874496 

37 

1.2574161 

79 

-0.36431 

120 

2.8563734 

161 

-3.244211 

202 

-6.144736 

38 

2.2250971 

80 

-5.434813 

121 

2.3255549 

162 

-2.227628 

203 

2.9630925 

39 

1.8653747 

81 

-3.942042 

122 

2.4607728 

163 

-2.560789 

204 

-2.089906 

40 

-4.762077 

82 

1.4118226 

123 

-1.592922 

164 

1.3499583 

205 

2.9363524 

41 

2.9624868 

83 

-2.081267 

124 

-0.678291 

165 

1.4051896 

206 

2.95151 

42 

2.9503679 

Table  4.6 


Combined  Array  One  Dimensional  Standardized  Response  Data 
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Source 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean 

Square 

F 

Value 

p- value 
Prob  <  F 

Model 

878.7585661 

41 

21.43313576 

10.84543387 

<  0.0001 

A 

3.792812756 

1 

3.792812756 

1.919210533 

0.1678 

B 

13.07239132 

1 

13.07239132 

6.614792961 

0.0110 

C 

5.018482374 

1 

5.018482374 

2.539414639 

0.1130 

D 

2.118686367 

1 

2.118686367 

1.072081712 

0.3020 

E 

59.59742067 

1 

59.59742067 

30.15703777 

<  0.0001 

F 

0.328422092 

1 

0.328422092 

0.166185673 

0.6841 

G 

0.521041485 

1 

0.521041485 

0.263653486 

0.6083 

H 

0.063147215 

1 

0.063147215 

0.031953278 

0.8584 

J 

21.27632813 

1 

21.27632813 

10.76608725 

0.0013 

K 

1.135396085 

1 

1.135396085 

0.574524572 

0.4496 

L 

4.774900156 

1 

4.774900156 

2.416159 

0.1220 

M 

559.7014493 

7 

79.9573499 

40.45941575 

<  0.0001 

BE 

11.23480116 

1 

11.23480116 

5.684949432 

0.0183 

BM 

44.58736527 

7 

6.36962361 

3.223108947 

0.0032 

EM 

42.77769292 

7 

6.111098989 

3.092292266 

0.0043 

JM 

51.3238605 

7 

7.331980072 

3.710073313 

0.0009 

J2 

47.93661496 

1 

47.93661496 

24.25652472 

<  0.0001 

Residual 

324.1026876 

164 

1.9762359 

Lack  of  Fit 

320.9456241 

157 

2.044239644 

4.532591004 

0.0199 

Pure  Error 

3.157063475 

7 

0.451009068 

Cor  Total 

1202.861254 

205 

Table  4.7  ANOVA  for  One  Dimensional  Standardized  Response 
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O  =  0.928732608  -  0.16093589  x  A 


+  0.287762309  x  B  +  0.18357479  x  C 
+  0.118276589  x  D  +  0.622239684  x  E 
+  0.047485806  x  F  -  0.059166689  x  G 
+  0.021716595  x  H  +  0.365644744  x  J 
+  0.076358805  x  K  -  0.160423161  x  L 
+  0.774824531  x  M[  1]  -  1.779421789  x  M[ 2] 

+  2.456865236  x  M[ 3]  +  1.464838509  x  M[ 4] 

+  1.22827625  x  M[ 5]  -  2.906496946  x  M[ 6] 

-  0.28650865  x  M[ 7]  -  0.29846376  x 

-  0.145106798  x  M[l]  -  0.609422884  x  BM[ 2] 

+  0.050601077  x  M[3]  +  1.104654031  x  BM[ 4] 

+  0.391108084  x  BM[5]  -  0.258359334  x  BM[ 6] 

+  0.165166983  x  M[7]  -  0.661393958  x  EM[  1] 

-  0.21250146  x  .EM  [2]  -  0.483520254  x  M[3] 

+  0.220174878  x  EM[ 4]  -  0.493187564  x  EM[5] 

+  0.435434061  x  £M[6]  +  0.235208103  x  EM[7] 

+  0.519721233  x  JM[  1]  -  0.185781319  x  JM[2] 

-  0.688586044  x  JM[3]  -  1.094824113  x  JM[ 4] 

+  0.459949215  x  JM[5]  +  0.782221794  x  JM[6] 

+  0.01039305  x  JM[ 7]  -  1.197693658  x  J2  (4.1) 


This  model  was  converted  using  the  techniques  from  section  3.4.1  into  an  esti¬ 
mated  mean  response  model  for  predicting  the  expected  value  of  the  mean  response 
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across  all  noise  categories.  Once  again,  the  observed  probabilities  of  occurence  given 
in  Table  3.6  were  used  within  Eq.  3.4  to  produce  a  reduced  estimated  mean  model 
for  the  one  dimensional  response  provided  in  Eq.  4.2.  Figures  containing  the  step- 
by-step  analysis  conducted  in  Excel®  are  included  in  App.  B,  Figures  B.l  and  B.2. 

E~z  y(d,  z)  =  1.091525694 

-  0.16093589  x  A 
+  0.373480157  x  B 
+  0.18357479  x  C 
+  0.118276589  x  D 
+  0.493179787  x  E 
+  0.047485806  x  F 

-  0.059166689  x  G 
+  0.021716595  x  H 
+  0.32925413  x  J 
+  0.076358805  x  K 

-  0.160423161  x  L 

-  0.29846376  x  BE 

-  1.197693658  x  J 2  (4.2) 

The  next  step  in  the  analysis  is  to  develop  a  variance  model  from  Eq.  4.1. 
Continuing  the  procedure  layed  out  in  section  3.4.2,  Eq.  3.7  is  used  to  generate  a 
variance  model  since  the  pz  is  known  for  each  category  of  the  noise.  Once  again, 
this  model  is  extremely  large  due  to  the  number  of  variance  and  covariance  terms 
generated  within  the  summations,  therefore  it  is  not  included  here  in  equation  form. 
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Variance  Model 

Estimate 

Interaction  (B) 

Interaction(E) 

Interaction(J) 

P 

Q 

m  r  11 

0.774824531 

-0.145106798 

-0.661393958 

0.519721233 

0.121359223 

0.878640777 

M[2] 

-1.779421789 

-0.609422884 

-0.21250146 

-0.185781319 

0.126213592 

0.873786408 

M  f  31 

2.456865236 

0.050601077 

-0.483520254 

-0.688586044 

0.13592233 

0.86407767 

M[4] 

1.464838509 

1.104654031 

0.220174878 

-1.094824113 

0.121359223 

0.878640777 

M[51 

1.22827625 

0.391108084 

-0.493187564 

0.459949215 

0.126213592 

0.873786408 

M  [6] 

-2.906496946 

-0.258359334 

0.435434061 

0.782221794 

0.116504854 

0.883495146 

M[71 

-0.28650865 

0.165166983 

0.235208103 

0.01039305 

0.121359223 

0.878640777 

Figure  4.12 


Elements  of  7  and  A  within  the  Variance  Model 


Squared  Terms  *PQ 
0.391828773 
4.239763716 
0.000209398 
1.487914038 
3.174797899 
0.169757375 
0.063153693 


Figure  4.13  Products  of  Squared  terms,  p,  and  q  within  first  summation  term  in 
Eq.  3.7 

However,  the  terms  of  the  variance  model  are  included  in  Figures  4.12,  4.13,  and 
4.14. 

The  values  in  Fig.  4.12  are  taken  directly  from  Eq.  4.1  and  are  the  elements 
of  7  and  A  in  Eq.  3.7.  They  are  appropriately  multiplied  by  the  control  variable 
levels,  then  the  entire  term  is  squared  before  being  multiplied  by  the  product  of  the 
appropriate  elements  of  the  P  and  Q  columns.  Recall  from  Eq.  2.13  multinomial 
variance  is  given  by  diagonal  elements,  pq ,  and  covariance  by  the  off-diagonal  terms, 
— piPj .  The  values  in  Fig.  4.13  are  the  results  of  these  computations  and  represent 
the  first  term  in  Eq.  3.7. 

The  double  sum  term  is  computed  in  a  similar  manner,  however  instead  of 
squaring  a  row’s  result,  it  is  multiplied  by  another  row’s  result.  This  product  is  then 
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Covariance  Terms  Matrix  (Upper  and  lower  combined) 

V(M[11) 

-0.364107918 

-0.002670326 

0.210925221 

0.315077462 

-0.069613445 

0.043454921 

V(M[2]) 

0.008982694 

-0.709530194 

-1.059887347 

0.234172287 

-0.146177771 

V(M[3]) 

-0.005203613 

-0.007773092 

0.001717393 

-0.001072051 

V(M[4]) 

0.613985474 

-0.1356544 

0.084679781 

V(M[5]) 

-0.20263885 

0.126493599 

V(M[6]) 

-0.027947588 

V(M[7]) 

Figure  4.14  Covariance  terms  summed  within  the  second  term  of  Eq.  3.7 


Parameter 

Type 

Test  Range 

Max  Score  Threshold 
Bin  Width  Indentify 
Window  Size 

Continuous 

Continuous 

Discrete 

[6,  14] 
[0.01,  0.1] 

[1,  9] 

Table  4.8  Variance  Model  Control  Factors  and  their  ranges 

multiplied  by  2  and  it’s  appropriate  covariance.  The  values  in  Fig.  4.14  represent 
these  covariance  products  and  their  sum  is  the  second  term  in  Eq.  3.7. 

The  final  term  is  simply  the  variance  from  the  model  and  is  found  in  the 
ANOVA  table,  4.7,  as  the  Mean  Square  of  Residual.  In  this  case  a2  =  1.9762359. 

All  the  components  of  the  variance  model  are  now  present,  so  all  that  remains 
is  to  minimize  the  sum  of  the  elements  in  Fig.  4.13  and  Fig.  4.14,  and  a2.  This  min¬ 
imization  is  subject  to  bounding  contstraints  on  the  ranges  of  the  control  variables 
present  in  the  variance  model.  These  variables  along  with  their  upper  and  lower 
bounds  are  restated  in  Table  4.8. 

Evaluation  of  the  minimization  was  conducted  using  an  Excel®  add-in  called 
What’s  Best!®  9.0  developed  by  LINDO  Systems,  Inc.  This  software  makes  a 
powerful  LP/IP  solver  available  within  a  spreadsheet.  It  is  therefore  ideal  to  solve 
the  mixed  integer  minimization  problem  presented  by  the  variance  model  developed 
above.  The  minimized  variance  returned  by  the  solver  was  11.54013698.  The  actual 
value  of  the  minimized  variance  doesn’t  convey  much  information  in  and  of  itself. 
What  is  important  is  the  levels  of  the  control  variables  that  produce  it.  These 


4-25 


Parameter 

Level 

Max  Score  Threshold 

6 

Bin  Width  Indentify 

0.1 

Window  Size 

4 

Tabic  4.9  Control  Factor  Levels  producing  Minimized  Variance 


Parameter 

Type 

Test  Range 

Dimension  Adjust 

Discrete 

[-2.  2] 

Bin  Width  SNR 

Continuous 

[0.01,  0.1] 

PT  SNR  Threshold 

Continuous 

[1.6] 

Smooth  Iterations  High 

Discrete 

[50,  150] 

Smooth  Iterations  Low 

Disrete 

[5,  45] 

Low  SNR 

Continuous 

[4,  14] 

Threshold  Both  Sides 

Categorical 

[0,  1] 

Clean  Signal 

Categorical 

[0,  1] 

Table  4.10  Estimated  Mean  Model  Control  Factors  and  their  ranges 

levels  represent  the  region  on  the  response  surface  where  variance  is  minimized,  and 
consistent  algorithm  performance  can  be  expected,  regardless  of  the  image  of  interest. 
The  control  variable  levels  producing  minimized  response  variance  are  presented  in 
Table  4.9. 

The  next  and  final  step  in  analysis  of  the  Combined  Array  is  to  maximize 
the  estimated  mean  response.  This  is  done  by  fixing  the  dispersion  factors  at  the 
levels  determined  above  to  minimize  response  variance  and  then  maximizing  Eq.  4.2 
by  manipulating  the  remaining  adjustment  factors.  This  maximization  was  done 
using  the  What’s  Best!®  add-in  described  earlier.  For  this  problem  the  remaining 
significant  controls  were  varied  across  their  respective  ranges  given  in  Table  4.10. 

The  solver  software  produced  a  maximized  estimated  mean  response  of  —6.3861. 
Recall  that  this  is  the  one  dimensional  standardized  response  and  as  such,  expresses 
the  summation  of  four  individual  standardized  responses  as  shown  in  Eq.  3.10.  So, 
even  though  a  negative  result  from  a  maximization  problem  seems  nonintuitive,  its 
underlying  cause  is  relatively  straightforward.  Since  each  of  the  responses  was  given 
equal  weight  in  the  objective  function,  none  is  allowed  to  excel  at  the  expense  of  the 
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Parameter 

Level 

Dimension  Adjust 

-2 

Max  Score  Threshold 

6 

Bin  Width  SNR 

0.1 

PT  SNR  Threshold 

6 

Bin  Width  Indentify 

0.1 

Smooth  Iterations  High 

150 

Smooth  Iterations  Low 

5 

Low  SNR 

14 

Window  Size 

4 

Threshold  Both  Sides 

1 

Clean  Signal 

0 

Table  4.11  Control  Factor  Levels  producing  Minimized  Variance  and  Optimized 
Response 

others.  Therefore,  each  response  settles  on  a  value  below  it’s  sample  mean  and  thus 
produces  a  negative  standardized  response.  Summing  four  negative  responses  to¬ 
gether  inevitably  results  in  a  negative  estimated  mean  response.  It  should  be  noted 
that  dividing  the  maximized  response  by  four  reveals  the  individual  responses  all 
contributing  ~-1.59  to  the  total.  In  other  words,  each  response  is  within  2  standard 
deviations  of  its  mean,  which  is  a  reasonable  expectation  given  the  equal  weighting 
between  them  and  the  constraint  to  the  minimized  variance  region  of  the  response 
surface. 

Finally,  the  control  variable  levels  producing  minimized  variance  and  subse¬ 
quently  optimized  response  are  summarized  in  Table  4.11. 


4-3  Verif cation  Experiment 

Control  factors  enabling  both  optimized  and  robust  algorithm  performance 
have  now  been  located  using  two  separate  RPD  techniques.  There  are  however,  a 
few  questions  that  remain: 

•  Is  one  technique  better  than  another  from  an  experimental  design  perspective 
(e.g.  design  matrix  size,  difficulty  of  analysis)? 
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•  Which  technique  produced  better  parameter  settings? 

•  Are  the  settings  from  either  or  both  techniques  better  than  those  suggested  by 

Johnson? 

Answering  the  first  question  is  a  somewhat  subjective  matter.  While  quantify¬ 
ing  the  design  size  is  objective  enough;  the  Crossed  Array  required  2240  replications 
while  the  Combined  Array  required  only  206,  this  is  only  a  portion  of  the  answer. 
Because  of  the  one-  [noise]  factor- at-a-time  nature  of  the  Crossed  Array  designs,  they 
will  always  be  larger  than  a  Combined  Array  design  of  the  same  resolution  for  a  par¬ 
ticular  system.  For  this  particular  analysis,  the  difference  in  number  of  replications 
was  not  of  particular  consequence,  since  each  replication  only  required  an  average 
of  ~21  seconds  to  complete.  So  even  2240  runs  could  be  completed  in  less  than  14 
hours.  However,  as  the  techniques  are  applied  to  other  algorithms  with  longer  per 
replication  execution  times,  design  size  becomes  more  important. 

Measuring  the  difficulty  of  analysis  is  where  the  first  question  becomes  more 
subjective.  Most  analysts  would  agree  the  Taguchi  methodology  is  generally  less  te¬ 
dious  and  therefore  more  expedient  with  regard  to  complete  the  analysis.  However, 
the  underlying  source  of  the  difference  in  required  analysis  effort  is  also  the  point 
of  greatest  criticism  for  Taguchi’s  methods.  Not  only  has  the  statistical  validity  of 
the  proposed  SNR’s  been  questioned,  but  they  are  also  the  point  within  the  analysis 
where  response  and  variance  are  confounded.  This  aggregation  precludes  the  devel¬ 
opment  of  separate  response  and  variance  models,  and  as  Myers  and  Montgomery 
[10]  point  out,  development  of  separate  models  often  provides  significant  insight  into 
system  processes. 

For  this  particular  analysis,  the  mixed  variable  nature  of  the  control  and  noise 
factors  required  specialized  techniques  for  the  Combined  Array  analysis,  whereas 
the  Crossed  Array  analysis  was  somewhat  indifferent  to  these  characteristics.  Once 
again,  as  these  techniques  are  applied  to  the  general  algorithm  with  more  standard 
control  and  noise  spaces,  the  Combined  Array  analysis  is  more  straightforward  in 
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Crossed  Array  Statistics  | 

Time 

TPF 

FPF 

TFP  | 

Image 

Mean 

Variance 

Mean 

Variance 

Mean 

Variance 

Mean 

Variance 

ARES 

1C 

1.2121 

0.0030 

1 

0 

0 

0 

1 

0 

ARES 

ID 

6.0014 

1.8244 

0 

0 

0 

0 

0 

0 

ARES 

IF 

3.0394 

0.0299 

0.4604 

3.17x  10~17 

0.0015 

4.84x  10-22 

0.9165 

0 

ARES 

2C 

2.2088 

0.1912 

1 

0 

0 

0 

1 

0 

ARES 

2D 

3.4996 

0.1342 

0.8235 

2  13-06 

0 

0 

1 

0 

ARES 

2F 

104.82 

66.1218 

0.9274 

0.0001 

0.0024 

2.88x  10-07 

0.7569 

0.0020 

ARES 

3F 

3.5923 

0.2642 

0.7137 

7.97x  10-06 

0.0004 

0 

0.9146 

7.71  xlO~08 

ARES 

4F 

2.2397 

0.0951 

0.7363 

8.26x  10-06 

0.0021 

1.14x  10-08 

0.7685 

6.81  xlO~05 

Total 

15.8262 

1158.68 

0.7077 

0.1010 

0.0008 

9.91x  10-07 

0.7946 

0.1005 

Table  4.12  Mean  and  Variances  from  Crossed  Array  Verification  Experiment 


nature;  effectively  closing  the  analysis  effort  gap  on  Crossed  Array  analysis.  Given 
these  analysis  attributes  combined  with  significant  difference  in  design  sizes,  the 
Combined  Array  is,  in  general,  better  than  the  Crossed  Array  with  regard  to  exper¬ 
imental  design. 

The  best  way  to  answer  two  remaining  questions  is  to  conduct  another  designed 
experiment.  However,  development  and  analysis  of  this  experiment  is  significantly 
less  involved  than  those  previously  discussed.  The  experiment  used  for  verifying 
algorithm  performance  at  the  suggested  levels  consisted  of  64  runs  at  each  of  three 
sets  of  suggested  control  settings.  Within  each  64-run  block,  each  noise  category  was 
tested  eight  times.  Eight  noise  categories  each  tested  eight  times  on  three  groups 
of  control  factor  settings  yields  192  total  runs.  The  entire  192-run  experiment  was 
randomized  to  satisfy  the  independence  assumption.  The  response  data  for  each 
run  was  collected  and  resorted  to  restore  the  control  setting  blocks  and  the  noise 
category  groupings.  Mean  and  variance  was  then  computed  for  each  technique’s 
suggested  settings  both  at  the  by  image  level  and  across  the  entire  noise  space. 

Table  4.12  gives  the  mean  and  variance  results  from  the  Crossed  Array  block 
for  each  response  both  by  image  and  as  an  aggregated  total  of  all  images.  Tables 
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Combined  Array  Statistics  | 

Time 

TPF 

FPF 

TFP  j 

Image 

Mean 

Variance 

Mean 

Variance 

Mean 

Variance 

Mean 

Variance 

ARES 

1C 

1.2078 

0.0028 

1 

0 

0 

0 

1 

0 

ARES 

ID 

6.4289 

4.7663 

0 

0 

0 

0 

0 

0 

ARES 

IF 

2.5197 

0.0760 

0.4195 

0 

0.0011 

0 

0.9295 

0 

ARES 

2C 

7.8425 

108.02 

1 

0 

0 

0 

1 

0 

ARES 

2D 

2.1147 

0.1772 

0.9023 

9.16x  10~°5 

0.0008 

5.0xl0'°9 

0.9664 

8.25X10'06 

ARES 

2F 

99.744 

0.9181 

0.8718 

0.0001 

0.0045 

2.57xl0~°7 

0.5710875 

0.0005 

ARES 

3F 

2.8082 

0.1066 

0.6883 

0.0005 

0.0013 

9.82xl0'°9 

0.717 

0.0004 

ARES 

4F 

1.8884 

0.0237 

0.8241 

1.56x  10~°5 

0.0055 

4.98xl0~°8 

0.5699 

0.0001 

Total 

15.569 

1045.9 

0.7133 

0.1059 

0.0017 

4.19x10'°® 

0.7192 

0.1048 

Table  4.13  Mean  and  Variances  from  Combined  Array  Verification  Experiment 


4.13  and  4.16  do  likewise  for  the  Combined  Array  and  Johnson’s  suggested  setting 
blocks,  respectively. 

Answering  the  second  question:  “Which  technique  produced  better  parameter 
settings?”  is  now  simply  a  matter  of  comparing  the  performance  statistics  in  Tables 
4.12  and  4.13.  However,  it  is  instructive  to  first  examine  the  differences  in  param¬ 
eter  settings  suggested  by  each  of  the  techniques.  Therefore,  a  complete  list  of  the 
suggested  parameters  for  each  technique  is  given  in  Table  4.14  where  differences  are 
highlighted  in  red. 

The  first  difference  is  in  ‘Bin  Width  SNR’  which  effects  the  smoothness  of  the 
signal  histogram.  The  smaller  bin  sizes  selected  by  the  Crossed  Array  produce  a 
less  smooth  histogram  and  reduce  the  SNR  required  for  a  map  to  be  nominated 
as  containing  potential  targets.  Next  is  ‘PT  SNR  Threshold’  which  draws  the  line 
separating  target  and  non-target  pixels  within  target  maps.  The  Crossed  Array 
selections  for  both  these  parameters  favor  a  more  liberal  approach  to  potential  target 
nomination. 
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Parameter 

Crossed  Array 
Level 

Combined  Array 
Level 

Dimension  Adjust 

-2 

-2 

Max  Score  Threshold 

6 

6 

Bin  Width  SNR 

0.06 

0.1 

PT  SNR  Threshold 

3.5 

6 

Bin  Width  Indentify 

0.1 

0.1 

Smooth  Iterations  High 

150 

150 

Smooth  Iterations  Low 

25 

5 

Low  SNR 

4 

14 

Window  Size 

5 

4 

Threshold  Both  Sides 

1 

1 

Clean  Signal 

1 

0 

Table  4.14  Control  Factor  Level  Differences  between  Crossed  and  Combined  Array 
Techniques 

The  four  remaining  parameters  are  all  related  in  that  the  first  three:  ‘Smooth 
Iterations  Low’,  ‘Low  SNR’,  and  ‘Window  Size’  are  only  used  if  ‘Clean  Signal’  is 
set  to  1.  Therefore,  the  ‘Clean  Signal’  setting  suggested  by  the  Combined  Array 
analysis  renders  the  other  three  parameters’  settings  irrelevant,  and  their  suggested 
settings  are  secondary  effects  from  other  parameters  influencing  them  through  the 
test  design  framework. 

Comparing  the  performance  statistics  from  the  two  RPD  techniques  reveals 
only  slight  differences  in  performance  in  any  one  response,  and  neither  technique 
outperformed  the  other  for  every  response.  The  Combined  Array  setting  was  an 
average  of  ~0.26  seconds  faster  and  produced  ~1%  greater  TPF  than  the  Crossed 
Array  setting.  However,  the  Crossed  Array  setting  produced  a  slightly  smaller  vari¬ 
ance  with  regard  to  TPF.  Additionally,  the  Crossed  Array  setting  outperformed  the 
Combined  Array  on  FPF  by  about  0.1%  and  TFP  by  almost  7%.  The  variance  of 
these  two  response  was  also  slightly  lower  with  the  Crossed  Array  than  the  Com¬ 
bined  Array  setting.  The  overall  differences  in  performance  between  the  two  sets 
of  parameter  settings  are  small  enough  that,  even  though  a  rigorous  multivariate 
analysis  was  not  performed,  they  can  be  said  to  be  insignificant.  However,  since 


4-31 


Parameter 

Johnson’s 

Level 

Dimension  Adjust 

0 

Max  Score  Threshold 

10 

Bin  Width  SNR 

0.05 

PT  SNR  Threshold 

2 

Bin  Width  Indentify 

0.05 

Smooth  Iterations  High 

100 

Smooth  Iterations  Low 

20 

Low  SNR 

10 

Window  Size 

3 

Threshold  Both  Sides 

0 

Clean  Signal 

1 

Table  4.15  Johnson’s  Suggested  Control  Factor  Levels 

the  second  question  from  the  previous  list  should  be  answered,  the  Time  difference 
was  transformed  into  a  percentage  to  provide  some  objective  method  of  comparison. 
The  Combined  Array  settings  were  ~1.6%  faster  than  the  Crossed  Array.  This  com¬ 
bined  with  the  1%  advantage  in  TPF  was  compared  to  the  ~7.1%  advantage  of  the 
Crossed  Array  from  FPF  and  TFP.  The  results  of  the  performance  comparison  show 
the  Crossed  Array  settings  provide  better,  more  robust  algorithm  performance  than 
the  Combined  Array  settings.  The  most  intuitive  reason  for  enhanced  performance 
from  the  Crossed  Array  settings  is  the  difference  in  design  sizes.  The  Crossed  Array’s 
design  is  a  full  order  of  magnitude  larger  than  the  Combined  Array  design  resulting 
in  enhanced  resolution  of  the  control  x  response  interaction. 

The  final  question  is  whether  or  not  the  parameters  suggested  by  the  RPD 
techniques  improve  performance  over  Johnson’s  suggested  settings.  The  settings 
suggested  by  Johnson  [5]  are  given  in  Table  4.15. 

Comparing  these  settings  to  those  suggested  by  the  RPDs,  Johnson’s  recom¬ 
mendations  match  those  of  the  RPD  for  only  one  control,  ‘Clean  Signal’.  Further¬ 
more,  this  setting  is  only  matched  by  the  Crossed  Array  and  it  is  one  of  two  binary 
variables,  making  the  intersection  likelihood  rather  high. 
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Johnson  Settings  Statistics 

Time 

TPF 

FPF 

TFP 

Image 

Mean 

Variance 

Mean 

Variance 

Mean 

Variance 

Mean 

Variance 

ARES 

1C 

1.5710 

0.1043 

1 

0 

0 

0 

1 

0 

ARES 

ID 

5.5502 

0.2338 

0.84 

0 

0.0023 

1.44x  10-07 

0.6125 

0.0015 

ARES 

IF 

4.2262 

0.5895 

0.9623 

1.27xl0~16 

0.0017 

4.84x  10-22 

0.9563 

2.54xl0~16 

ARES 

2C 

3.5078 

0.0782 

1 

0 

0.0056 

1.24x  10-06 

0 

0 

ARES 

2D 

2.9759 

0.1232 

0.9085 

6.85xlO~06 

0.0003 

5.14x  10-08 

0.9889 

7.07xl0-05 

ARES 

2F 

122.61 

0.4541 

0.9704 

3.43xlO“06 

0.0025 

1.28xlO'06 

0.7772 

0.0053 

ARES 

3F 

5.1454 

0.4610 

0.8456 

5.65xlO~os 

0.0004 

1.25xl0-°9 

0.9242 

4.16xlO~06 

ARES 

4F 

27.538 

0.3373 

0.8441 

0 

0.0037 

1.15xl0“°6 

0.7272 

0.0029 

Total 

21.640 

1542.2 

0.9214 

0.0044 

0.0021 

3.71xl0~°6 

0.7483 

0.0995 

Table  4.16  Mean  and  Variances  from  Johnson  Verification  Experiment 


Comparing  the  statistics  in  Table  4.16  with  those  in  Tables  4.12  and  4.13  shows 
significant  differences  in  mean  response  between  Johnson  and  the  RPDs.  First, 
Johnson’s  settings  yield  an  average  Time  response  approximately  6  seconds  higher 
than  those  of  the  RPDs  indicating  an  improvement  in  average  algorithm  run  time 
with  RPD.  However,  Johnson’s  settings  produced  an  average  TPF  over  20%  higher 
than  either  of  the  RPDs.  FPF  was  slightly  higher  using  Johnson’s  settings,  but  all 
of  the  FPFs  are  very  small  with  virtually  no  variance.  Finally,  Johnson’s  settings 
produced  an  average  TFP  lower  than  the  Crossed  Array,  but  higher  than  Combined 
Array  settings. 

These  results  suggest  that  while  the  RPDs  did  not  enhance  performance  in 
each  response  category,  neither  did  they  degrade  overall  algorithm  performance. 
The  differences  in  performance  observed  in  the  Time  and  TPF  responses  are  most 
likely  the  result  of  the  objective  functions  given  in  Eq.’s  3.10  and  3.11.  Recall  these 
equations  give  equal  weight  to  each  of  the  four  responses.  These  weightings  are 
key  to  driving  the  algorithm  toward  a  response (s)  of  interest.  For  this  particular 
analysis,  no  user  input  was  available  in  terms  of  required  analysis  time  or  minimum 
TPF,  so  each  of  the  responses  was  treated  as  equally  important.  From  the  results 
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shown  in  Table  4.16  it  seems  apparent  that  Johnson  was  somewhat  more  concerned 
with  TPF  than  Time.  In  other  words,  his  objective  function,  had  it  been  explicitly 
stated,  would  have  a  larger  weight  on  TPF  than  on  the  Time  response. 

In  conclusion,  the  relative  equality  of  performance  demonstrated  by  the  sug¬ 
gested  settings  from  the  RPD  techniques  combined  with  the  large  variation  in  design 
size  and  the  additional  system  level  insight  gained  by  generating  mean  and  variance 
models  leads  to  selection  of  the  Combined  Array  RSM  technique  as  preferable  to 
Taguchi’s  Crossed  Array  method. 
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5.  Conclusions 

This  final  chapter  reviews  the  previous  four,  summarizes  the  contributions  of  the 
research  efforts,  and  suggests  possible  future  research  areas. 

5.1  Review 

Chapter  1  began  by  introducing  remote  sensing,  specifically  the  proliferation 
of  hyperspectral  technology  and  the  analysis  problems  it  creates.  A  previously  de¬ 
veloped  method,  AutoGAD,  is  introduced  as  potential  piece  of  future  analysis 
techniques,  but  has  not  yet  evolved  into  a  real  time  analysi  tool  ready  for  deploy¬ 
ment.  Design  of  Experiments,  specifically  Robust  Parameter  Design,  is  proposed  as  a 
technique  for  improving  AutoGAD’s  capability.  Finally,  the  location  of  robust  op¬ 
erating  parameters  via  two  distinct  RPD  methods  was  presented  as  the  overarching 
research  goal. 

Chapter  2  provided  a  short  description  of  hyperspectral  imagery  as  well  as 
common  techniques  for  analysis  of  HSI.  This  was  followed  by  an  overview  of  the 
AutoGAD  algorithm  developed  by  Johnson  for  autonomous  detection  of  anomalous 
pixels  within  HSI.  Robust  Parameter  Design’s  history,  application,  and  techniques 
were  discussed.  This  discussion  included  development  of  the  mathematical  basis  for 
two  separate  RPD  techniques:  Taguchi’s  Method,  or  Crossed  Arrays;  and  Response 
Surface  Methods,  or  Combined  Arrays. 

Chapter  3  cast  AutoGAD  as  a  RPD  problem.  Inputs  (controls  and  noise) 
were  developed  as  well  as  responses  to  quantify  algorithm  performance.  These  vari¬ 
ables  were  then  employed  to  develop  RPDs  for  AutoGAD  using  both  Crossed  and 
Combined  Array  approaches.  Finally,  the  dimensionality  of  the  response  space  was 
reduced  using  standardization  and  a  simple  linear  program. 
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Chapter  4  provided  detailed  analysis  of  the  experimental  results.  Both  RPD 
techniques  were  described  in  detail  with  regard  to  their  application  to  AutoGAD. 
The  results  from  both  designed  experiments  were  presented  along  with  analysis  of 
these  results.  The  suggested  robust  parameter  settings  were  provided  and  then 
compared  using  a  verification  experiment.  This  experiment  included  a  comparison 
of  both  RPD  techniques  to  Johnson’s  suggested  settings.  Finally,  the  results  of 
the  verification  were  presented  and  techniques’  performances  were  compared  to  each 
other  and  to  the  performance  of  Johnson’s  settings. 

5.2  Research  Contributions 

The  list  given  below  summarizes  the  contributions  to  the  body  of  knowledge 
made  by  this  research: 

•  Developed  Methodology  for  evaluating  new  evolutions  of  AutoGAD. 

•  RPD  can  provide  better  settings,  or  at  least  match  the  performance  of  settings 
suggested  by  algorithm  creators  by  giving  flexibility  to  the  end-user  through 
objective  function  weight  manipulation. 

•  RPD  has  been  shown  as  a  viable  technique  for  enhancing  anomaly  detection 
algorithm  performance.  It  is  reasonable  to  assume  the  techniques  are  extensible 
to  other  types  of  algorithms. 

•  Conducted  head-to-head  comparison  of  two  distinct  RPD  techniques;  forming 
and  supporting  conclusions  about  which  technique  is  better  suited  to  problems 
of  similar  type  and  size. 

•  Developed  a  method  for  reducing  the  dimensionality  of  the  response  space  for 
multi-objective  problems.  Specifically,  standardizing  response  data  to  enable 
creation  of  a  simple,  one- dimensional  objective  function  where  all  responses  of 
interest  become  terms  of  a  linear  combination. 
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5.3  Future  Research 

•  Conduct  a  large  scale  validation  experiment  using  different  imagery  collected 
at  different  times,  using  different  sensors  and  targets/backgrounds. 

•  The  low  R2  of  the  Combined  Array  model  suggests  misspecihcation.  Employ 
different  model  generation  techniques  to  locate  robust  parameter  settings,  i.e. 
Neural  Networks. 

•  Develop  a  more  sophisticated  heuristic  for  selection  of  significant  parameters. 
These  heuristics  can  be  used  within  both  techniques  to  decide  which  parameters 
will  adjust  mean  response  and  which  will  be  used  to  reduce  response  variation. 
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Appendix  A.  Additional  Data  Tables  and 


Analysis  from  the  Crossed  Array  Design 
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II  "'  M  |M  |i 

74 1  2|  14 1  0.1 

Ijagr^  i^KE 

0.9436  -0.50481  0.637574  -0.36731  0.987046 

0.7004  -0.46316  0.34448  -0.33619  0.390883 

0.8313  -0.46913  0.694895  -0.10054  0.711762 

0.1801  2.392995  0.703006  1.202229  -0.88454 

0.8876  -0.46064  -0.60753  -0.42067  0.849771 

0.1263  -0.46833  0.319335  0.690905  -1.01642 

1|  0.01 1  50 1  5 

-jgfeF;  llji^Tjfl 

89  2  6  0.1 

90  -2  6  0.1 

91  -2  6  0.1 

4  1  0  0  1  4.0849  0.8432  0.0001  0.9963  -0.45875  0.361785  -0.42956  1.116231 

4  9  0  0  1  75.2561  0.9271  0.1394  0.0401  1.372946  0.588635  5.764131  -1.22773 

4  1  0  0  1  5.0678  0.8696  0.0046  0.4965  -0.43346  0.433165  -0.22948  -0.10894 

1|  0.01|  150 1  5 

0.8268  2.576806  0.335017  -0.37176  0.700731 

0.9474  -0.4819  0.468315  -0.35397  0.996361 

1|  0.01 1  50 1  5 

98  -2  6  0.1 

99  -2  14  0.1 

1|  0.01 1  50 1  5 

101|  2|  14 1  0.1 

111  -2  6  0.1 

112  2  14  0.1 

1|  0.01 1  50 1  5 

114  -2  6  0.1 

4|  9|  0|  0|  1|  73.8819 1  0.8768|  0.0038|  0.6126|  1.337579|  0.452633|  -0.26505|  0.175657| 

115  2  14  0.1 

9|  5|  0|  0|  1|  5.1011|  0.9428|  0.0017|  0.9539|  -0.4326|  0.631085|  -0.35842|  1.012294| 

BBR3EMZ3  BEZUSS  EEESEDElSilS]  1 

124  0  10  0.055 

6  0.055  100  25 

9  5  0  0  1  1.2696  1  0 

1  -0.53121  0.785743  -0.43401  1.125301 

125  0  10  0.055 

3.5  0.01  100  25 

9  5  0  0  1  3.6442  0.7799  0.0159 

0.3872  -0.4701  0.190633  0.272953  -0.37687 

126  0  10  0.055 

3.5  0.1  100  25 

9  5  0  0  1  3.4184  0.6218  0.0017 

0.6016  -0.47591  -0.23684  -0.35842  0.148692 

127  0  10  0.055 

3.5  0.055  50  25 

9  5  0  0  1  2.7417  0.6034  0.0114 

0.3686  -0.49332  -0.28659  0.07287  -0.42247 

128  0  10  0.055 

3.5  0.055  150  25 

9  5  0  0  1  5.1318  0.8571  0.0038 

0.5412  -0.43181  0.399368  -0.26505  0.000632 

129  0  10  0.055 

3.5  0.055  100  5 

9  5  0  0  1  2.2621  1  0 

1  -0.50567  0.785743  -0.43401  1.125301 

130  0  10  0.055 

3.5  0.055  100  45 

9  5  0  0  1  3.4719  0.8826  0.0018 

0.9474  -0.47453  0.468315  -0.35397  0.996361 

131  0  10  0.055 

3.5  0.055  100  25 

4  5  0  0  1  157.2884  0  0 

0  3.484173  -1.91807  -0.43401  -1.32603 

132  0  10  0.055 

3.5  0.055  100  25 

14  5  0  0  1  9.2282  0  0 

0  -0.32638  -1.91807  -0.43401  -1.32603 

133  0  10  0.055 

3.5  0.055  100  25 

9  1  0  0  1  25.6972  1  0 

1  0.097471  0.785743  -0.43401  1.125301 

134  0  10  0.055 

3.5  0.055  100  25 

9  9  0  0  1  6.0156  0.8368  0.0026 

0.6709  -0.40906  0.34448  -0.3184  0.318569 

135  0  10  0.055 

3.5  0.055  100  25 

9  5  0  0  1  1.7539  1  0 

1  -0.51875  0.785743  -0.43401  1.125301 

136  0  10  0.055 

3.5  0.055  100  25 

9  5  0  0  1  6.4026  0.6489  0.0364 

0.1162  -0.3991  -0.16357  1.184444  -1.04118 
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9|  5|  0|  0|  1|  5.4437 1  0.8509|  0.004l|  0.5249|  -0.42378|  0.382604|  -0.2517l|  -0.03932| 

142 |  2|  6|  0.1 

147  2  6  0.1 

148  2  14  0.1 

4|  9|  l|  1|  1|  2.5616|  0.7515|  0.0008|  0.907l|  -0.49796|  0.113845|  -0.39844|  0.897572| 

153 1  -2|  14 1  0.1 

159 |  2|  6|  0.1 

4|  1|  l|  1|  1|  3.63221  0.78161  0.012l|  0.7446|  -0.4704l|  0.195229|  0.103994|  0.499232| 

161 |  -2|  6|  0.1 

6  0.01  150  5 

4|  9|  l|  1|  1|  2.5953|  0.7778|  0.0026|  0.7557|  -0.49709|  0.184955|  -0.3184|  0.52644l| 

1  0.01  150  5 

mm 

171 1  2|  14 1  0.1 

tel  ii  Baffin 

176 |  -2|  14 |  0.1 

6|  0.01|  150 1  5 

181 |  -2|  6|  0.1 

6  0.01  50  5 

1  0.01  50  5 

9|  5|  l|  1|  1|  1.6862|  0.8924|  0.0005|  0.9776|  -0.52049|  0.494812|  -0.41178|  1.07039l| 

9|  5|  l|  1|  1|  3.2967|  0.4604|  0.0018|  0.9028|  -0.47904|  -0.67324|  -0.35397|  0.887032| 

192 |  0|  10|  0.1 

9  5  1  1  1  98.6556  0.8869  0.0051  0.5492  1.975168  0.479941  -0.20725  0.020242 

9  5  1  1  1  3.5696  0.8538  0.0001  0.9964  -0.47202  0.390445  -0.42956  1.116476 
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196  0  10  0.055 

3.5  0.1  100  25 

9  5  1  1  1  1.439  1  0  1  -0.52685  0.785743  -0.43401  1.125301 

197  0  10  0.055 

3.5  0.055  50  25 

9  5  1  1  1  1.1777  1  0  1  -0.53358  0.785743  -0.43401  1.125301 

198  0  10  0.055 

3.5  0.055  150  25 

9  5  1  1  1  4.2403  0.8571  0.0038  0.5412  -0.45475  0.399368  -0.26505  0.000632 

199  0  10  0.055 

3.5  0.055  100  5 

9  5  1  1  1  1.9931  0.9545  0.0815  0.1128  -0.51259  0.662719  3.189724  -1.04952 

200  0  10  0.055 

3.5  0.055  100  45 

9  5  1  1  1  3.8547  0.8385  0.002  0.6888  -0.46468  0.349077  -0.34508  0.362448 

201  0  10  0.055 

3.5  0.055  100  25 

4  5  1  1  1  3.2041  0.5267  0.0006  0.8876  -0.48142  -0.49397  -0.40733  0.849771 

202  0  10  0.055 

3.5  0.055  100  25 

14  5  1  1  1  5.3075  0.824  0.0018  0.9461  -0.42729  0.309871  -0.35397  0.993174 

203  0  10  0.055 

3.5  0.055  100  25 

9  1  1  1  1  5.0074  0.8304  0.0181  0.6977  -0.43501  0.327176  0.370772  0.384264 

204  0  10  0.055 

3.5  0.055  100  25 

9  9  1  1  1  137.5998  0.9576  0.0008  0.9206  2.977457  0.671101  -0.39844  0.930665 

205  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  1  5.9362  0.9899  0.0082  0.452  -0.41111  0.758434  -0.06941  -0.21803 

206  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  1  3.1345  1  0  1  -0.48321  0.785743  -0.43401  1.125301 

207  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  1  3.6366  0.7513  0.0643  0.1198  -0.47029  0.113304  2.424961  -1.03236 

208  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  1  2.754  0.8825  0.0018  0.9483  -0.49301  0.468044  -0.35397  0.998567 
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Table  A.l 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  1)  in  the  Crossed  Array  Design 
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Crossed  Array  Inner  Array  Design  Points  and  Responses  (Block  2) 


Std  Run 

< 

E 

Q 

Max  Score 

Bin  width  SNR 

PT  SNR  thresh 

Bin  Width  Ident 

Smooth  iter  hi 

Smooth  iter  lo 

Low  SNR 

Window  Size 

Threshold  Both  Sides 

Clean  Signal 

Image  (Noise) 

Time 

TPF 

FPF 

TFP 

Std  Time 

Std  TPF 

Std  FPF 

Std  TFP 

281 

2 

6 

0.01 

6 

0.1 

50 

5 

4 

1 

1 

0 

2 

2.6128 

1 

0 

1 

-0.49664 

0.785743 

-0.43401 

1.125301 

282 

2 

6 

0.1 

1 

0.1 

50 

45 

14 

9 

1 

0 

2 

2.22 

0.4655 

0.0018 

0.9044 

-0.50675 

-0.65945 

-0.35397 

0.890954 

283 

2 

14 

0.01 

6 

0.1 

50 

5 

14 

9 

1 

0 

2 

3.7059 

0 

0 

0 

-0.46851 

-1.91807 

-0.43401 

-1.32603 

284 

2 

14 

0.1 

1 

0.1 

150 

45 

4 

9 

1 

0 

2 

4.3897 

0 

0.0011 

0 

-0.45091 

-1.91807 

-0.3851 

-1.32603 

285 

2 

14 

0.01 

1 

0.1 

50 

5 

4 

9 

1 

0 

2 

2.745 

1 

0 

1 

-0.49324 

0.785743 

-0.43401 

1.125301 

286 

-2 

6 

0.01 

1 

0.1 

150 

5 

14 

9 

1 

0 

2 

3.7632 

0.4572 

0.0001 

0.9937 

-0.46703 

-0.68189 

-0.42956 

1.109857 

287 

2 

6 

0.1 

6 

0.1 

150 

5 

14 

9 

1 

0 

2 

7.7178 

0 

0.0011 

0 

-0.36526 

-1.91807 

-0.3851 

-1.32603 

288 

2 

14 

0.1 

6 

0.1 

150 

5 

4 

1 

1 

0 

2 

3.0899 

0 

0 

0 

-0.48436 

-1.91807 

-0.43401 

-1.32603 

289 

2 

6 

0.01 

6 

0.01 

50 

45 

4 

9 

1 

0 

2 

65.6367 

0.8052 

0.0023 

0.6359 

1.125376 

0.25904 

-0.33174 

0.232772 

290 

2 

6 

0.01 

6 

0.01 

150 

5 

14 

1 

1 

0 

2 

119.1528 

0.9102 

0.0026 

0.7119 

2.502695 

0.54294 

-0.3184 

0.419073 

291 

-2 

14 

0.01 

1 

0.1 

150 

5 

4 

1 

1 

0 

2 

2.412 

1 

0 

1 

-0.50181 

0.785743 

-0.43401 

1.125301 

292 

-2 

6 

0.1 

1 

0.01 

50 

5 

14 

1 

1 

0 

2 

2.1214 

1 

0 

1 

-0.50929 

0.785743 

-0.43401 

1.125301 

293 

-2 

14 

0.1 

1 

0.1 

50 

5 

14 

9 

1 

0 

2 

122.9508 

0 

0 

0 

2.600442 

-1.91807 

-0.43401 

-1.32603 

294 

2 

14 

0.1 

6 

0.01 

150 

45 

4 

9 

1 

0 

2 

1.6176 

0.8636 

0.0005 

0.9788 

-0.52225 

0.416942 

-0.41178 

1.073333 

295 

-2 

6 

0.01 

6 

0.01 

150 

45 

14 

9 

1 

0 

2 

4.2794 

0.8621 

0 

1 

-0.45375 

0.412887 

-0.43401 

1.125301 

296 

2 

14 

0.01 

1 

0.01 

150 

45 

4 

1 

1 

0 

2 

3.2134 

0.848 

0.0003 

0.9894 

-0.48118 

0.374763 

-0.42067 

1.099317 

297 

-2 

14 

0.01 

6 

0.01 

150 

45 

14 

1 

1 

0 

2 

2.9714 

0.9871 

0.0215 

0.5964 

-0.48741 

0.750864 

0.521946 

0.135945 

298 

2 

14 

0.01 

6 

0.1 

150 

45 

14 

1 

1 

0 

2 

4.3496 

0.6959 

0.0013 

0.7203 

-0.45194 

-0.03649 

-0.37621 

0.439664 

299 

2 

6 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

2 

1.157 

1 

0.0018 

0 

-0.53411 

0.785743 

-0.35397 

-1.32603 

300 

-2 

6 

0.1 

1 

0.01 

150 

45 

4 

9 

1 

0 

2 

5.9068 

0 

0.0024 

0 

-0.41186 

-1.91807 

-0.3273 

-1.32603 

301 

-2 

6 

0.1 

1 

0.1 

150 

5 

4 

1 

1 

0 

2 

2.1707 

1 

0 

1 

-0.50802 

0.785743 

-0.43401 

1.125301 

302 

2 

6 

0.01 

6 

0.01 

150 

5 

4 

9 

1 

0 

2 

1.8361 

0.9538 

0.0544 

0.1574 

-0.51663 

0.660827 

1.984778 

-0.94019 

303 

2 

14 

0.01 

1 

0.01 

150 

5 

14 

9 

1 

0 

2 

1.243 

1 

0 

1 

-0.5319 

0.785743 

-0.43401 

1.125301 

304 

-2 

6 

0.01 

1 

0.1 

150 

45 

14 

1 

1 

0 

2 

5.1157 

0 

0 

0 

-0.43223 

-1.91807 

-0.43401 

-1.32603 

305 

-2 

14 

0.01 

1 

0.01 

150 

45 

4 

9 

1 

0 

2 

3.3712 

1 

0.004 

0 

-0.47712 

0.785743 

-0.25616 

-1.32603 

306 

-2 

6 

0.01 

6 

0.1 

150 

45 

4 

9 

1 

0 

2 

2.2273 

1 

0 

1 

-0.50656 

0.785743 

-0.43401 

1.125301 

307 

2 

6 

0.01 

1 

0.01 

50 

5 

14 

9 

1 

0 

2 

108.4569 

0.8433 

0.0001 

0.9878 

2.227419 

0.362055 

-0.42956 

1.095394 

308 

-2 

6 

0.1 

6 

0.1 

150 

45 

14 

9 

1 

0 

2 

3.9371 

0.6359 

0.0618 

0.1101 

-0.46256 

-0.19872 

2.313804 

-1.05614 

309 

-2 

14 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

2 

1.8516 

1 

0 

1 

-0.51623 

0.785743 

-0.43401 

1.125301 

310 

-2 

6 

0.1 

6 

0.01 

150 

45 

4 

1 

1 

0 

2 

26.539 

0.8758 

0.0163 

0.3358 

0.119136 

0.449929 

0.290738 

-0.50287 

311 

2 

14 

0.1 

1 

0.01 

50 

45 

4 

9 

1 

0 

2 

1.9706 

1 

0 

1 

-0.51317 

0.785743 

-0.43401 

1.125301 

312 

-2 

14 

0.1 

6 

0.01 

50 

45 

14 

9 

1 

0 

2 

25.7818 

1 

0 

1 

0.099649 

0.785743 

-0.43401 

1.125301 

313 

-2 

14 

0.1 

6 

0.1 

50 

5 

14 

1 

1 

0 

2 

5.2815 

0.168 

0 

1 

-0.42796 

-1.46383 

-0.43401 

1.125301 

314 

2 

6 

0.01 

1 

0.01 

50 

45 

14 

1 

1 

0 

2 

5.1643 

0.7486 

0.0003 

0.9424 

-0.43097 

0.106004 

-0.42067 

0.984104 

315 

2 

6 

0.1 

6 

0.1 

150 

45 

4 

1 

1 

0 

2 

1.723 

1 

0 

1 

-0.51954 

0.785743 

-0.43401 

1.125301 

316 

-2 

14 

0.1 

6 

0.01 

150 

5 

14 

9 

1 

0 

2 

100.2439 

0.8537 

0.0045 

0.5668 

2.016045 

0.390175 

-0.23392 

0.063386 

317 

-2 

14 

0.01 

1 

0.1 

50 

45 

14 

9 

1 

0 

2 

1.3849 

1 

0 

1 

-0.52824 

0.785743 

-0.43401 

1.125301 

318 

-2 

14 

0.01 

6 

0.01 

50 

5 

4 

9 

1 

0 

2 

4.2783 

0.8105 

0.0023 

0.6018 

-0.45378 

0.27337 

-0.33174 

0.149182 

319 

2 

14 

0.1 

6 

0.1 

50 

45 

4 

1 

1 

0 

2 

1.4815 

1 

0 

1 

-0.52576 

0.785743 

-0.43401 

1.125301 

320 

-2 

6 

0.01 

6 

0.1 

50 

45 

14 

1 

1 

0 

2 

26.4651 

0.8742 

0.0112 

0.4204 

0.117234 

0.445603 

0.063977 

-0.29549 

321 

-2 

6 

0.1 

1 

0.1 

50 

45 

4 

1 

1 

0 

2 

5.5558 

0 

0 

0 

-0.4209 

-1.91807 

-0.43401 

-1.32603 

322 

2 

14 

0.1 

6 

0.01 

50 

5 

14 

1 

1 

0 

2 

25.1201 

0 

0 

0 

0.082619 

-1.91807 

-0.43401 

-1.32603 

323 

-2 

6 

0.01 

1 

0.01 

50 

5 

4 

1 

1 

0 

2 

24.172 

0 

0 

0 

0.058218 

-1.91807 

-0.43401 

-1.32603 

324 

-2 

6 

0.1 

6 

0.1 

50 

5 

4 

9 

1 

0 

2 

99.8286 

0.8537 

0.0011 

0.8434 

2.005357 

0.390175 

-0.3851 

0.741423 

325 

2 

14 

0.1 

1 

0.01 

150 

45 

14 

1 

1 

0 

2 

2.3906 

1 

0.0049 

0 

-0.50236 

0.785743 

-0.21614 

-1.32603 

326 

2 

14 

0.01 

1 

0.1 

50 

5 

14 

1 

1 

0 

2 

3.3019 

0 

0 

0 

-0.47891 

-1.91807 

-0.43401 

-1.32603 

327 

-2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

2 

2.6561 

0.8289 

0.0204 

0.3189 

-0.49553 

0.32312 

0.473036 

-0.5443 

328 

2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

2 

5.8809 

0 

0 

0 

-0.41253 

-1.91807 

-0.43401 

-1.32603 

329 

0 

6 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

2 

1.7554 

0.9121 

0.0008 

0.9663 

-0.51871 

0.548077 

-0.39844 

1.042691 

330 

0 

14 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

2 

1.9408 

0.956 

0.0833 

0.1141 

-0.51394 

0.666775 

3.269758 

-1.04633 

331 

0 

10 

0.01 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

2 

6.0519 

0 

0 

0 

-0.40813 

-1.91807 

-0.43401 

-1.32603 

332 

0 

10 

0.1 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

2 

6.0791 

0.7892 

0.0003 

0.9481 

-0.40743 

0.215778 

-0.42067 

0.998077 

333 

0 

10 

0.055 

1 

0.055 

100 

25 

9 

5 

1 

0 

2 

4.2988 

0 

0.0004 

0 

-0.45325 

-1.91807 

-0.41622 

-1.32603 

334 

0 

10 

0.055 

6 

0.055 

100 

25 

9 

5 

1 

0 

2 

5.0323 

0 

0 

0 

-0.43437 

-1.91807 

-0.43401 

-1.32603 

335 

0 

10 

0.055 

3.5 

0.01 

100 

25 

9 

5 

1 

0 

2 

1.5463 

0.9617 

0.0843 

0.1141 

-0.52409 

0.682187 

3.314221 

-1.04633 

336 

0 

10 

0.055 

3.5 

0.1 

100 

25 

9 

5 

1 

0 

2 

4.6678 

0.8825 

0.0018 

0.9483 

-0.44375 

0.468044 

-0.35397 

0.998567 

337 

0 

10 

0.055 

3.5 

0.055 

50 

25 

9 

5 

1 

0 

2 

1.5835 

1 

0 

1 

-0.52313 

0.785743 

-0.43401 

1.125301 

338 

0 

10 

0.055 

3.5 

0.055 

150 

25 

9 

5 

1 

0 

2 

9.0039 

0 

0 

0 

-0.33216 

-1.91807 

-0.43401 

-1.32603 

339 

0 

10 

0.055 

3.5 

0.055 

100 

5 

9 

5 

1 

0 

2 

3.2605 

1 

0 

1 

-0.47997 

0.785743 

-0.43401 

1.125301 

340 

0 

10 

0.055 

3.5 

0.055 

100 

45 

9 

5 

1 

0 

2 

1.6644 

1 

0 

1 

-0.52105 

0.785743 

-0.43401 

1.125301 

341 

0 

10 

0.055 

3.5 

0.055 

100 

25 

4 

5 

1 

0 

2 

100.2243 

0.8673 

0.0008 

0.8809 

2.015541 

0.426946 

-0.39844 

0.833348 

342 

0 

10 

0.055 

3.5 

0.055 

100 

25 

14 

5 

1 

0 

2 

3.1422 

1 

0.0047 

0 

-0.48302 

0.785743 

-0.22503 

-1.32603 

343 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

1 

1 

0 

2 

1.3617 

1 

0.0013 

0 

-0.52884 

0.785743 

-0.37621 

-1.32603 

344 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

9 

1 

0 

2 

3.4701 

0.8509 

0.0036 

0.5547 

-0.47458 

0.382604 

-0.27394 

0.033725 

345 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

2 

3.7963 

0.6218 

0.0031 

0.4512 

-0.46618 

-0.23684 

-0.29617 

-0.21999 

A-5 


A-6 


0.055  _ 3.5  0.055  100  5 _ 9 

0.055  _ 3.5  0.055  100  45 _ 9  ~ 

0.055  _ 3.5  0.055  100  25 _ 4  ~ 

0.055  _ 3.5  0.055  100  25 _ 14  ~ 

0.055  _ 3.5  0.055  100  25 _ 9  ~ 

0.055  _ 3.5  0.055  100  25 _ 9  ~ 

0.055  _ 3.5  0.055  100  25 _ 9  ~ 

0.055  _ 3.5  0.055  100  25 _ 9  ~ 

0.055  _ 3.5  0.055  100  25 _ 9  ~ 

0.055  _ 3.5  0.055  100  25 _ 9  ~ 

0.055  _ 3.5  0.055  100  25 _ 9  ~ 

0.055  3.5  0.055  100  25  9  ~ 


2  99.3914  0.8869 

2  75.8783  0.9153  ~ 

2  4.0714  0  ~ 

2  2.8914  1  ~ 

2  5.3446  0.8283  ~ 

2  5.8588  0.7892  ~ 

2  19.6568  0  ~ 

2  165.9218  0  ~ 


Table  A. 2 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  2)  in  the  Crossed  Array  Design 


A-8 


Crossed  Array  Inner  Array  Design  Points  and  Responses  (Block  3) 


Std  Run 

< 

E 

5 

Max  Score 

Bin  width  SNR 

PT  SNR  thresh 

Bin  Width  Ident 

Smooth  iter  hi 

Smooth  iter  lo 

Low  SNR 

Window  Size 

Threshold  Both  Sides 

Clean  Signal 

Image  (Noise) 

Time 

TPF 

FPF 

TFP 

Std  Time 

Std  TPF 

Std  FPF 

Std  TFP 

561 

2 

6 

0.01 

6 

0.1 

50 

5 

4 

1 

1 

0 

3 

77.4162 

0.8408 

0.0019 

0.6984 

1.428539 

0.355295 

-0.34953 

0.38598 

562 

2 

6 

0.1 

1 

0.1 

50 

45 

14 

9 

1 

0 

3 

1.4742 

1 

0 

1 

-0.52594 

0.785743 

-0.43401 

1.125301 

563 

2 

14 

0.01 

6 

0.1 

50 

5 

14 

9 

1 

0 

3 

5.7049 

0.9428 

0.0017 

0.9539 

-0.41706 

0.631085 

-0.35842 

1.012294 

564 

2 

14 

0.1 

1 

0.1 

150 

45 

4 

9 

1 

0 

3 

27.4505 

0.765 

0.007 

0.5534 

0.142595 

0.150346 

-0.12277 

0.030538 

565 

2 

14 

0.01 

1 

0.1 

50 

5 

4 

9 

1 

0 

3 

105.2287 

0 

0 

0 

2.144337 

-1.91807 

-0.43401 

-1.32603 

566 

-2 

6 

0.01 

1 

0.1 

150 

5 

14 

9 

1 

0 

3 

7.1295 

0 

0.001 

0 

-0.3804 

-1.91807 

-0.38955 

-1.32603 

567 

2 

6 

0.1 

6 

0.1 

150 

5 

14 

9 

1 

0 

3 

2.3958 

0.7971 

0.0121 

0.3583 

-0.50223 

0.237139 

0.103994 

-0.44772 

568 

2 

14 

0.1 

6 

0.1 

150 

5 

4 

1 

1 

0 

3 

98.5794 

0.8838 

0.0014 

0.8141 

1.973207 

0.471559 

-0.37176 

0.669599 

569 

2 

6 

0.01 

6 

0.01 

50 

45 

4 

9 

1 

0 

3 

97.9672 

0.9675 

0.0111 

0.477 

1.957451 

0.697869 

0.059531 

-0.15674 

570 

2 

6 

0.01 

6 

0.01 

150 

5 

14 

1 

1 

0 

3 

8.3949 

0 

0 

0 

-0.34783 

-1.91807 

-0.43401 

-1.32603 

571 

-2 

14 

0.01 

1 

0.1 

150 

5 

4 

1 

1 

0 

3 

169.4264 

0 

0 

0 

3.796563 

-1.91807 

-0.43401 

-1.32603 

572 

-2 

6 

0.1 

1 

0.01 

50 

5 

14 

1 

1 

0 

3 

4.4235 

0 

0.0015 

0 

-0.45004 

-1.91807 

-0.36731 

-1.32603 

573 

-2 

14 

0.1 

1 

0.1 

50 

5 

14 

9 

1 

0 

3 

2.2925 

1 

0 

1 

-0.50488 

0.785743 

-0.43401 

1.125301 

574 

2 

14 

0.1 

6 

0.01 

150 

45 

4 

9 

1 

0 

3 

2.2911 

1 

0.0059 

0 

-0.50492 

0.785743 

-0.17168 

-1.32603 

575 

-2 

6 

0.01 

6 

0.01 

150 

45 

14 

9 

1 

0 

3 

2.8793 

1 

0 

1 

-0.48978 

0.785743 

-0.43401 

1.125301 

576 

2 

14 

0.01 

1 

0.01 

150 

45 

4 

1 

1 

0 

3 

3.2397 

0 

0 

0 

-0.48051 

-1.91807 

-0.43401 

-1.32603 

577 

-2 

14 

0.01 

6 

0.01 

150 

45 

14 

1 

1 

0 

3 

4.2313 

0.5993 

0.0001 

0.9941 

-0.45499 

-0.29768 

-0.42956 

1.110838 

578 

2 

14 

0.01 

6 

0.1 

150 

45 

14 

1 

1 

0 

3 

2.695 

1 

0 

1 

-0.49453 

0.785743 

-0.43401 

1.125301 

579 

2 

6 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

3 

5.6902 

0.017 

0.0144 

0.0048 

-0.41744 

-1.87211 

0.206259 

-1.31426 

580 

-2 

6 

0.1 

1 

0.01 

150 

45 

4 

9 

1 

0 

3 

3.7824 

0.6667 

0.0608 

0.1166 

-0.46654 

-0.11544 

2.269341 

-1.0402 

581 

-2 

6 

0.1 

1 

0.1 

150 

5 

4 

1 

1 

0 

3 

4.1154 

0 

0.0018 

0 

-0.45797 

-1.91807 

-0.35397 

-1.32603 

582 

2 

6 

0.01 

6 

0.01 

150 

5 

4 

9 

1 

0 

3 

122.4541 

0.9541 

0.0006 

0.9327 

2.587659 

0.661638 

-0.40733 

0.960326 

583 

2 

14 

0.01 

1 

0.01 

150 

5 

14 

9 

1 

0 

3 

5.8947 

0 

0 

0 

-0.41218 

-1.91807 

-0.43401 

-1.32603 

584 

-2 

6 

0.01 

1 

0.1 

150 

45 

14 

1 

1 

0 

3 

3.8842 

0.5175 

0.0038 

0.5441 

-0.46392 

-0.51885 

-0.26505 

0.007741 

585 

-2 

14 

0.01 

1 

0.01 

150 

45 

4 

9 

1 

0 

3 

1.9123 

0.9928 

0.0242 

0.5661 

-0.51467 

0.766275 

0.641996 

0.06167 

586 

-2 

6 

0.01 

6 

0.1 

150 

45 

4 

9 

1 

0 

3 

2.3652 

0.9632 

0.002 

0.9274 

-0.50301 

0.686242 

-0.34508 

0.947334 

587 

2 

6 

0.01 

1 

0.01 

50 

5 

14 

9 

1 

0 

3 

1.5521 

1 

0 

1 

-0.52394 

0.785743 

-0.43401 

1.125301 

588 

-2 

6 

0.1 

6 

0.1 

150 

45 

14 

9 

1 

0 

3 

69.8842 

0.5906 

0.0006 

0.8417 

1.234692 

-0.3212 

-0.40733 

0.737256 

589 

-2 

14 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

3 

104.1027 

0 

0 

0 

2.115357 

-1.91807 

-0.43401 

-1.32603 

590 

-2 

6 

0.1 

6 

0.01 

150 

45 

4 

1 

1 

0 

3 

99.982 

0.9184 

0.009 

0.417 

2.009305 

0.565111 

-0.03384 

-0.30382 

591 

2 

14 

0.1 

1 

0.01 

50 

45 

4 

9 

1 

0 

3 

1.2651 

1 

0 

1 

-0.53133 

0.785743 

-0.43401 

1.125301 

592 

-2 

14 

0.1 

6 

0.01 

50 

45 

14 

9 

1 

0 

3 

9.9644 

0 

0 

0 

-0.30744 

-1.91807 

-0.43401 

-1.32603 

593 

-2 

14 

0.1 

6 

0.1 

50 

5 

14 

1 

1 

0 

3 

119.9208 

0.9167 

0.0163 

0.2862 

2.52246 

0.560515 

0.290738 

-0.62446 

594 

2 

6 

0.01 

1 

0.01 

50 

45 

14 

1 

1 

0 

3 

33.6544 

0 

0 

0 

0.302262 

-1.91807 

-0.43401 

-1.32603 

595 

2 

6 

0.1 

6 

0.1 

150 

45 

4 

1 

1 

0 

3 

2.3869 

1 

0.0059 

0 

-0.50246 

0.785743 

-0.17168 

-1.32603 

596 

-2 

14 

0.1 

6 

0.01 

150 

5 

14 

9 

1 

0 

3 

4.1394 

0 

0.0018 

0 

-0.45735 

-1.91807 

-0.35397 

-1.32603 

597 

-2 

14 

0.01 

1 

0.1 

50 

45 

14 

9 

1 

0 

3 

3.771 

1 

0 

1 

-0.46683 

0.785743 

-0.43401 

1.125301 

598 

-2 

14 

0.01 

6 

0.01 

50 

5 

4 

9 

1 

0 

3 

2.0637 

0.8264 

0.0057 

0.5613 

-0.51077 

0.31636 

-0.18057 

0.049903 

599 

2 

14 

0.1 

6 

0.1 

50 

45 

4 

1 

1 

0 

3 

4.1226 

0.8521 

0.0224 

0.1447 

-0.45778 

0.385848 

0.561962 

-0.97132 

600 

-2 

6 

0.01 

6 

0.1 

50 

45 

14 

1 

1 

0 

3 

116.515 

0.9697 

0.0204 

0.3895 

2.434807 

0.703817 

0.473036 

-0.37123 

601 

-2 

6 

0.1 

1 

0.1 

50 

45 

4 

1 

1 

0 

3 

3.4906 

0.9326 

0.0006 

0.9854 

-0.47405 

0.603506 

-0.40733 

1.089511 

602 

2 

14 

0.1 

6 

0.01 

50 

5 

14 

1 

1 

0 

3 

6.4134 

0 

0.0024 

0 

-0.39883 

-1.91807 

-0.3273 

-1.32603 

603 

-2 

6 

0.01 

1 

0.01 

50 

5 

4 

1 

1 

0 

3 

160.9852 

0 

0 

0 

3.579316 

-1.91807 

-0.43401 

-1.32603 

604 

-2 

6 

0.1 

6 

0.1 

50 

5 

4 

9 

1 

0 

3 

27.7538 

0.8774 

0.0188 

0.3077 

0.150401 

0.454255 

0.401896 

-0.57175 

605 

2 

14 

0.1 

1 

0.01 

150 

45 

14 

1 

1 

0 

3 

2.1248 

0.9452 

0.0014 

0.9469 

-0.5092 

0.637574 

-0.37176 

0.995135 

606 

2 

14 

0.01 

1 

0.1 

50 

5 

14 

1 

1 

0 

3 

11.6765 

0.9612 

0.0035 

0.6804 

-0.26337 

0.680835 

-0.27839 

0.341857 

607 

-2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

3 

3.8909 

0.8825 

0.0018 

0.9483 

-0.46375 

0.468044 

-0.35397 

0.998567 

608 

2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

3 

118.4215 

0.9611 

0.0058 

0.6198 

2.483874 

0.680564 

-0.17612 

0.193306 

609 

0 

6 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

3 

2.6702 

1 

0.0055 

0 

-0.49516 

0.785743 

-0.18946 

-1.32603 

610 

0 

14 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

3 

5.1728 

0.017 

0.0144 

0.0048 

-0.43076 

-1.87211 

0.206259 

-1.31426 

611 

0 

10 

0.01 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

3 

5.5052 

0.8826 

0.0018 

0.9474 

-0.4222 

0.468315 

-0.35397 

0.996361 

612 

0 

10 

0.1 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

3 

2.7053 

1 

0 

1 

-0.49426 

0.785743 

-0.43401 

1.125301 

613 

0 

10 

0.055 

1 

0.055 

100 

25 

9 

5 

1 

0 

3 

115.0784 

0.9701 

0.039 

0.1754 

2.397834 

0.704899 

1.300048 

-0.89606 

614 

0 

10 

0.055 

6 

0.055 

100 

25 

9 

5 

1 

0 

3 

3.6904 

0.7688 

0.0035 

0.6833 

-0.46891 

0.160621 

-0.27839 

0.348965 

615 

0 

10 

0.055 

3.5 

0.01 

100 

25 

9 

5 

1 

0 

3 

3.2744 

1 

0 

1 

-0.47961 

0.785743 

-0.43401 

1.125301 

616 

0 

10 

0.055 

3.5 

0.1 

100 

25 

9 

5 

1 

0 

3 

114.7225 

0.9548 

0.006 

0.6107 

2.388674 

0.66353 

-0.16723 

0.170999 

617 

0 

10 

0.055 

3.5 

0.055 

50 

25 

9 

5 

1 

0 

3 

2.9222 

0.7711 

0.0015 

0.8366 

-0.48868 

0.166839 

-0.36731 

0.724754 

618 

0 

10 

0.055 

3.5 

0.055 

150 

25 

9 

5 

1 

0 

3 

2.6493 

1 

0.0048 

0 

-0.4957 

0.785743 

-0.22059 

-1.32603 

619 

0 

10 

0.055 

3.5 

0.055 

100 

5 

9 

5 

1 

0 

3 

6.8595 

0.8385 

0.0018 

0.7068 

-0.38735 

0.349077 

-0.35397 

0.406572 

620 

0 

10 

0.055 

3.5 

0.055 

100 

45 

9 

5 

1 

0 

3 

27.6752 

0.8758 

0.013 

0.3873 

0.148378 

0.449929 

0.14401 

-0.37663 

621 

0 

10 

0.055 

3.5 

0.055 

100 

25 

4 

5 

1 

0 

3 

123.7927 

0.9194 

0.0405 

0.2526 

2.62211 

0.567815 

1.366742 

-0.70682 

622 

0 

10 

0.055 

3.5 

0.055 

100 

25 

14 

5 

1 

0 

3 

2.1513 

0.8965 

0.0007 

0.97 

-0.50852 

0.505898 

-0.40288 

1.051761 

623 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

1 

1 

0 

3 

3.0834 

0.8189 

0.0091 

0.7677 

-0.48453 

0.296082 

-0.0294 

0.555857 

624 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

9 

1 

0 

3 

164.4685 

0 

0 

0 

3.668964 

-1.91807 

-0.43401 

-1.32603 

625 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

3 

3.0653 

1 

0.004 

0 

-0.485 

0.785743 

-0.25616 

-1.32603 

A-9 


ini 


1HE23H! 

3EJE1J 


_ _ o.oi  loo  ; 

10  0.055  _ 3.5  0.1  100  25 _ 9  " 

10  0.055  _ 3.5  0.055  50 _ 25 _ 9  ~ 

10  0.055  _ 3.5  0.055  150  25 _ 9  ~ 

10  0.055  _ 3.5  0.055  100  5 _ 9  ” 

10  0.055  _ 3.5  0.055  100  45 _ 9  " 

10  0.055  _ 3.5  0.055  100  25 _ 4  " 

10  0.055  _ 3.5  0.055  100  25 _ 14  _ 

10  0.055  _ 3.5  0.055  100  25 _ 9  ~ 

10  0.055  _ 3.5  0.055  100  25 _ 9  ” 

10  0.055  _ 3.5  0.055  100  25 _ 9  " 

10  0.055  _ 3.5  0.055  100  25 _ 9  ~ 

10  0.055  _ 3.5  0.055  100  25 _ 9  ~ 

10  0.055  3.5  0.055  100  25  9  ~ 


3  26.4592 

3  1.2421  ~ 

3  2.2951  ~ 

3  2.5377  ~ 

3  2.3706  ~ 

3  8.1166  ~ 

3  2.6126  ~ 

3  64.9343  ~ 

3  28.7078  ~ 

3  2.5593  ~ 

3  100.8161  ~ 
3  78.6975  ~ 

3  5.169  ~ 
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Table  A. 3 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  3)  in  the  Crossed  Array  Design 


A-12 


Crossed  Array  Inner  Array  Design  Points  and  Responses  (Block  4) 


Std  Run 

< 

E 

5 

Max  Score 

Bin  width  SNR 

PT  SNR  thresh 

Bin  Width  Ident 

Smooth  iter  hi 

Smooth  iter  lo 

Low  SNR 

Window  Size 

Threshold  Both  Sides 

Clean  Signal 

Image  (Noise) 

Time 

TPF 

FPF 

TFP 

Std  Time 

Std  TPF 

Std  FPF 

Std  TFP 

841 

2 

6 

0.01 

6 

0.1 

50 

5 

4 

1 

1 

0 

4 

3.9916 

0.8825 

0.0018 

0.9483 

-0.46116 

0.468044 

-0.35397 

0.998567 

842 

2 

6 

0.1 

1 

0.1 

50 

45 

14 

9 

1 

0 

4 

15.3932 

0 

0 

0 

-0.16772 

-1.91807 

-0.43401 

-1.32603 

843 

2 

14 

0.01 

6 

0.1 

50 

5 

14 

9 

1 

0 

4 

4.02 

0 

0 

0 

-0.46042 

-1.91807 

-0.43401 

-1.32603 

844 

2 

14 

0.1 

1 

0.1 

150 

45 

4 

9 

1 

0 

4 

3.3424 

0.8489 

0.0007 

0.974 

-0.47786 

0.377196 

-0.40288 

1.061566 

845 

2 

14 

0.01 

1 

0.1 

50 

5 

4 

9 

1 

0 

4 

2.6941 

0.8596 

0.0004 

0.9808 

-0.49455 

0.406127 

-0.41622 

1.078235 

846 

-2 

6 

0.01 

1 

0.1 

150 

5 

14 

9 

1 

0 

4 

6.6086 

0.9777 

0.0081 

0.43 

-0.3938 

0.725448 

-0.07386 

-0.27196 

847 

2 

6 

0.1 

6 

0.1 

150 

5 

14 

9 

1 

0 

4 

62.9869 

0.8408 

0.0077 

0.3587 

1.057179 

0.355295 

-0.09164 

-0.44674 

848 

2 

14 

0.1 

6 

0.1 

150 

5 

4 

1 

1 

0 

4 

2.6022 

0.9818 

0.0166 

0.6421 

-0.49691 

0.736533 

0.304077 

0.247971 

849 

2 

6 

0.01 

6 

0.01 

50 

45 

4 

9 

1 

0 

4 

100.31 

0.9679 

0.0699 

0.0991 

2.017746 

0.69895 

2.673954 

-1.0831 

850 

2 

6 

0.01 

6 

0.01 

150 

5 

14 

1 

1 

0 

4 

99.968 

0.9677 

0.0556 

0.1209 

2.008945 

0.69841 

2.038133 

-1.02966 

851 

-2 

14 

0.01 

1 

0.1 

150 

5 

4 

1 

1 

0 

4 

19.6047 

0 

0 

0 

-0.05933 

-1.91807 

-0.43401 

-1.32603 

852 

-2 

6 

0.1 

1 

0.01 

50 

5 

14 

1 

1 

0 

4 

165.754 

0 

0 

0 

3.702048 

-1.91807 

-0.43401 

-1.32603 

853 

-2 

14 

0.1 

1 

0.1 

50 

5 

14 

9 

1 

0 

4 

3.2081 

0.5203 

0.0004 

0.9167 

-0.48132 

-0.51128 

-0.41622 

0.921105 

854 

2 

14 

0.1 

6 

0.01 

150 

45 

4 

9 

1 

0 

4 

18.0492 

0.8921 

0.0003 

0.9658 

-0.09936 

0.494001 

-0.42067 

1.041465 

855 

-2 

6 

0.01 

6 

0.01 

150 

45 

14 

9 

1 

0 

4 

113.8874 

0.8886 

0.0072 

0.466 

2.367182 

0.484538 

-0.11387 

-0.18371 

856 

2 

14 

0.01 

1 

0.01 

150 

45 

4 

1 

1 

0 

4 

5.4242 

0 

0 

0 

-0.42429 

-1.91807 

-0.43401 

-1.32603 

857 

-2 

14 

0.01 

6 

0.01 

150 

45 

14 

1 

1 

0 

4 

4.551 

0.9663 

0.0053 

0.8741 

-0.44676 

0.694624 

-0.19835 

0.816679 

858 

2 

14 

0.01 

6 

0.1 

150 

45 

14 

1 

1 

0 

4 

3.2885 

1 

0.036 

0 

-0.47925 

0.785743 

1.166659 

-1.32603 

859 

2 

6 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

4 

1.4883 

1 

0 

1 

-0.52558 

0.785743 

-0.43401 

1.125301 

860 

-2 

6 

0.1 

1 

0.01 

150 

45 

4 

9 

1 

0 

4 

66.2345 

0.7922 

0.0021 

0.6559 

1.140761 

0.22389 

-0.34064 

0.281799 

861 

-2 

6 

0.1 

1 

0.1 

150 

5 

4 

1 

1 

0 

4 

9.6321 

0 

0 

0 

-0.31599 

-1.91807 

-0.43401 

-1.32603 

862 

2 

6 

0.01 

6 

0.01 

150 

5 

4 

9 

1 

0 

4 

3.418 

1 

0.0062 

0 

-0.47592 

0.785743 

-0.15834 

-1.32603 

863 

2 

14 

0.01 

1 

0.01 

150 

5 

14 

9 

1 

0 

4 

2.2854 

0.9548 

0.0824 

0.1123 

-0.50507 

0.66353 

3.229741 

-1.05074 

864 

-2 

6 

0.01 

1 

0.1 

150 

45 

14 

1 

1 

0 

4 

176.7845 

0 

0 

0 

3.985935 

-1.91807 

-0.43401 

-1.32603 

865 

-2 

14 

0.01 

1 

0.01 

150 

45 

4 

9 

1 

0 

4 

2.633 

1 

0.0045 

0 

-0.49612 

0.785743 

-0.23392 

-1.32603 

866 

-2 

6 

0.01 

6 

0.1 

150 

45 

4 

9 

1 

0 

4 

182.3727 

0 

0 

0 

4.129756 

-1.91807 

-0.43401 

-1.32603 

867 

2 

6 

0.01 

1 

0.01 

50 

5 

14 

9 

1 

0 

4 

6.8347 

0 

0 

0 

-0.38798 

-1.91807 

-0.43401 

-1.32603 

868 

-2 

6 

0.1 

6 

0.1 

150 

45 

14 

9 

1 

0 

4 

1.2688 

1 

0.0554 

0 

-0.53123 

0.785743 

2.029241 

-1.32603 

869 

-2 

14 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

4 

3.4215 

0.8432 

0.0001 

0.9963 

-0.47583 

0.361785 

-0.42956 

1.116231 

870 

-2 

6 

0.1 

6 

0.01 

150 

45 

4 

1 

1 

0 

4 

1.2605 

1 

0 

1 

-0.53144 

0.785743 

-0.43401 

1.125301 

871 

2 

14 

0.1 

1 

0.01 

50 

45 

4 

9 

1 

0 

4 

2.9762 

0.6959 

0.0011 

0.7518 

-0.48729 

-0.03649 

-0.3851 

0.516881 

872 

-2 

14 

0.1 

6 

0.01 

50 

45 

14 

9 

1 

0 

4 

29.0999 

0.768 

0.0057 

0.6183 

0.185045 

0.158458 

-0.18057 

0.189629 

873 

-2 

14 

0.1 

6 

0.1 

50 

5 

14 

1 

1 

0 

4 

2.1795 

0.4518 

0 

0.9979 

-0.50779 

-0.69649 

-0.43401 

1.120153 

874 

2 

6 

0.01 

1 

0.01 

50 

45 

14 

1 

1 

0 

4 

5.568 

0.168 

0 

1 

-0.42058 

-1.46383 

-0.43401 

1.125301 

875 

2 

6 

0.1 

6 

0.1 

150 

45 

4 

1 

1 

0 

4 

1.992 

1 

0 

1 

-0.51262 

0.785743 

-0.43401 

1.125301 

876 

-2 

14 

0.1 

6 

0.01 

150 

5 

14 

9 

1 

0 

4 

4.6216 

0.709 

0.0029 

0.5118 

-0.44494 

-0.00107 

-0.30507 

-0.07144 

877 

-2 

14 

0.01 

1 

0.1 

50 

45 

14 

9 

1 

0 

4 

4.8661 

0.6032 

0 

1 

-0.43865 

-0.28713 

-0.43401 

1.125301 

878 

-2 

14 

0.01 

6 

0.01 

50 

5 

4 

9 

1 

0 

4 

1.4645 

1 

0 

1 

-0.52619 

0.785743 

-0.43401 

1.125301 

879 

2 

14 

0.1 

6 

0.1 

50 

45 

4 

1 

1 

0 

4 

1.9263 

0.8846 

0.0111 

0.4326 

-0.51431 

0.473722 

0.059531 

-0.26558 

880 

-2 

6 

0.01 

6 

0.1 

50 

45 

14 

1 

1 

0 

4 

34.078 

0 

0 

0 

0.313164 

-1.91807 

-0.43401 

-1.32603 

881 

-2 

6 

0.1 

1 

0.1 

50 

45 

4 

1 

1 

0 

4 

3.5922 

0.8483 

0.0003 

0.9894 

-0.47143 

0.375574 

-0.42067 

1.099317 

882 

2 

14 

0.1 

6 

0.01 

50 

5 

14 

1 

1 

0 

4 

120.417 

0.9167 

0.0163 

0.2862 

2.535231 

0.560515 

0.290738 

-0.62446 

883 

-2 

6 

0.01 

1 

0.01 

50 

5 

4 

1 

1 

0 

4 

2.9448 

0 

0 

0 

-0.4881 

-1.91807 

-0.43401 

-1.32603 

884 

-2 

6 

0.1 

6 

0.1 

50 

5 

4 

9 

1 

0 

4 

4.7105 

0.6468 

0.0002 

0.9896 

-0.44265 

-0.16924 

-0.42512 

1.099807 

885 

2 

14 

0.1 

1 

0.01 

150 

45 

14 

1 

1 

0 

4 

3.7521 

0.9428 

0.0022 

0.9411 

-0.46732 

0.631085 

-0.33619 

0.980917 

886 

2 

14 

0.01 

1 

0.1 

50 

5 

14 

1 

1 

0 

4 

3.1061 

1 

0.004 

0 

-0.48395 

0.785743 

-0.25616 

-1.32603 

887 

-2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

4 

1.6386 

1 

0 

1 

-0.52171 

0.785743 

-0.43401 

1.125301 

888 

2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

4 

4.1442 

0 

0.0012 

0 

-0.45723 

-1.91807 

-0.38065 

-1.32603 

889 

0 

6 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

4 

2.0581 

1 

0 

1 

-0.51092 

0.785743 

-0.43401 

1.125301 

890 

0 

14 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

4 

1.4019 

1 

0 

1 

-0.52781 

0.785743 

-0.43401 

1.125301 

891 

0 

10 

0.01 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

4 

11.4418 

0 

0 

0 

-0.26941 

-1.91807 

-0.43401 

-1.32603 

892 

0 

10 

0.1 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

4 

8.9668 

0 

0 

0 

-0.33311 

-1.91807 

-0.43401 

-1.32603 

893 

0 

10 

0.055 

1 

0.055 

100 

25 

9 

5 

1 

0 

4 

1.6439 

0.9607 

0.0537 

0.1641 

-0.52158 

0.679483 

1.953654 

-0.92376 

894 

0 

10 

0.055 

6 

0.055 

100 

25 

9 

5 

1 

0 

4 

10.9482 

0 

0 

0 

-0.28212 

-1.91807 

-0.43401 

-1.32603 

895 

0 

10 

0.055 

3.5 

0.01 

100 

25 

9 

5 

1 

0 

4 

10.7227 

0 

0.0032 

0 

-0.28792 

-1.91807 

-0.29173 

-1.32603 

896 

0 

10 

0.055 

3.5 

0.1 

100 

25 

9 

5 

1 

0 

4 

3.815 

1 

0.0074 

0 

-0.4657 

0.785743 

-0.10498 

-1.32603 

897 

0 

10 

0.055 

3.5 

0.055 

50 

25 

9 

5 

1 

0 

4 

25.7475 

1 

0 

1 

0.098766 

0.785743 

-0.43401 

1.125301 

898 

0 

10 

0.055 

3.5 

0.055 

150 

25 

9 

5 

1 

0 

4 

101.636 

0.9677 

0.0584 

0.1156 

2.051873 

0.69841 

2.16263 

-1.04265 

899 

0 

10 

0.055 

3.5 

0.055 

100 

5 

9 

5 

1 

0 

4 

104.5729 

0 

0 

0 

2.127459 

-1.91807 

-0.43401 

-1.32603 

900 

0 

10 

0.055 

3.5 

0.055 

100 

45 

9 

5 

1 

0 

4 

4.0571 

0 

0.0015 

0 

-0.45947 

-1.91807 

-0.36731 

-1.32603 

901 

0 

10 

0.055 

3.5 

0.055 

100 

25 

4 

5 

1 

0 

4 

3.9238 

0.6218 

0.0022 

0.5362 

-0.4629 

-0.23684 

-0.33619 

-0.01162 

902 

0 

10 

0.055 

3.5 

0.055 

100 

25 

14 

5 

1 

0 

4 

3.6008 

0.8439 

0.0003 

0.9891 

-0.47121 

0.363677 

-0.42067 

1.098581 

903 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

1 

1 

0 

4 

4.8751 

0.8032 

0.0157 

0.7073 

-0.43842 

0.253632 

0.26406 

0.407797 

904 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

9 

1 

0 

4 

119.3926 

0.9107 

0.0094 

0.4091 

2.508866 

0.544292 

-0.01606 

-0.32319 

905 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

4 

79.2803 

0.7195 

0.0492 

0.1055 

1.476515 

0.027323 

1.75357 

-1.06741 
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Table  A. 4 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  4)  in  the  Crossed  Array  Design 


A-16 


Crossed  Array  Inner  Array  Design  Points  and  Responses  (Block  5) 


Std  Run 

< 

E 

5 

Max  Score 

Bin  width  SNR 

PT  SNR  thresh 

Bin  Width  Ident 

Smooth  iter  hi 

Smooth  iter  lo 

Low  SNR 

Window  Size 

Threshold  Both  Sides 

Clean  Signal 

Image  (Noise) 

Time 

TPF 

FPF 

TFP 

Std  Time 

Std  TPF 

Std  FPF 

Std  TFP 

1121 

2 

6 

0.01 

6 

0.1 

50 

5 

4 

1 

1 

0 

5 

3.9219 

0.6218 

0.002 

0.5585 

-0.46295 

-0.23684 

-0.34508 

0.04304 

1122 

2 

6 

0.1 

1 

0.1 

50 

45 

14 

9 

1 

0 

5 

1.7126 

1 

0 

1 

-0.51981 

0.785743 

-0.43401 

1.125301 

1123 

2 

14 

0.01 

6 

0.1 

50 

5 

14 

9 

1 

0 

5 

5.5819 

0 

0 

0 

-0.42023 

-1.91807 

-0.43401 

-1.32603 

1124 

2 

14 

0.1 

1 

0.1 

150 

45 

4 

9 

1 

0 

5 

4.0001 

1 

0 

1 

-0.46094 

0.785743 

-0.43401 

1.125301 

1125 

2 

14 

0.01 

1 

0.1 

50 

5 

4 

9 

1 

0 

5 

113.6599 

0 

0 

0 

2.361327 

-1.91807 

-0.43401 

-1.32603 

1126 

-2 

6 

0.01 

1 

0.1 

150 

5 

14 

9 

1 

0 

5 

3.2772 

0 

0 

0 

-0.47954 

-1.91807 

-0.43401 

-1.32603 

1127 

2 

6 

0.1 

6 

0.1 

150 

5 

14 

9 

1 

0 

5 

12.4979 

0 

0 

0 

-0.24223 

-1.91807 

-0.43401 

-1.32603 

1128 

2 

14 

0.1 

6 

0.1 

150 

5 

4 

1 

1 

0 

5 

118.9498 

0 

0 

0 

2.49747 

-1.91807 

-0.43401 

-1.32603 

1129 

2 

6 

0.01 

6 

0.01 

50 

45 

4 

9 

1 

0 

5 

6.8325 

0.4142 

0.0008 

0.9474 

-0.38804 

-0.79815 

-0.39844 

0.996361 

1130 

2 

6 

0.01 

6 

0.01 

150 

5 

14 

1 

1 

0 

5 

6.0578 

0.5191 

0.0014 

0.61 

-0.40798 

-0.51452 

-0.37176 

0.169283 

1131 

-2 

14 

0.01 

1 

0.1 

150 

5 

4 

1 

1 

0 

5 

2.6431 

0.8281 

0.0038 

0.612 

-0.49586 

0.320957 

-0.26505 

0.174186 

1132 

-2 

6 

0.1 

1 

0.01 

50 

5 

14 

1 

1 

0 

5 

3.0092 

1 

0 

1 

-0.48644 

0.785743 

-0.43401 

1.125301 

1133 

-2 

14 

0.1 

1 

0.1 

50 

5 

14 

9 

1 

0 

5 

6.3506 

0.7792 

0.0129 

0.7305 

-0.40044 

0.18874 

0.139564 

0.464668 

1134 

2 

14 

0.1 

6 

0.01 

150 

45 

4 

9 

1 

0 

5 

2.2043 

0.9415 

0.0012 

0.9516 

-0.50715 

0.62757 

-0.38065 

1.006656 

1135 

-2 

6 

0.01 

6 

0.01 

150 

45 

14 

9 

1 

0 

5 

90.9647 

0.9711 

0.0566 

0.122 

1.777231 

0.707603 

2.082596 

-1.02696 

1136 

2 

14 

0.01 

1 

0.01 

150 

45 

4 

1 

1 

0 

5 

65.4847 

0 

0 

0 

1.121464 

-1.91807 

-0.43401 

-1.32603 

1137 

-2 

14 

0.01 

6 

0.01 

150 

45 

14 

1 

1 

0 

5 

3.0239 

0.9857 

0.0196 

0.6194 

-0.48606 

0.747078 

0.437466 

0.192326 

1138 

2 

14 

0.01 

6 

0.1 

150 

45 

14 

1 

1 

0 

5 

4.1034 

0.4847 

0.0004 

0.8681 

-0.45828 

-0.60753 

-0.41622 

0.801971 

1139 

2 

6 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

5 

8.1723 

0 

0 

0 

-0.35356 

-1.91807 

-0.43401 

-1.32603 

1140 

-2 

6 

0.1 

1 

0.01 

150 

45 

4 

9 

1 

0 

5 

64.2518 

0.8153 

0.0044 

0.4885 

1.089733 

0.286348 

-0.23837 

-0.12855 

1141 

-2 

6 

0.1 

1 

0.1 

150 

5 

4 

1 

1 

0 

5 

2.9749 

0.4199 

0.0009 

0.9416 

-0.48732 

-0.78274 

-0.39399 

0.982143 

1142 

2 

6 

0.01 

6 

0.01 

150 

5 

4 

9 

1 

0 

5 

2.8322 

1 

0 

1 

-0.49099 

0.785743 

-0.43401 

1.125301 

1143 

2 

14 

0.01 

1 

0.01 

150 

5 

14 

9 

1 

0 

5 

65.6207 

0.9579 

0.1516 

0.0378 

1.124964 

0.671912 

6.306579 

-1.23337 

1144 

-2 

6 

0.01 

1 

0.1 

150 

45 

14 

1 

1 

0 

5 

2.3507 

0.9928 

0.031 

0.5073 

-0.50339 

0.766275 

0.944344 

-0.08247 

1145 

-2 

14 

0.01 

1 

0.01 

150 

45 

4 

9 

1 

0 

5 

4.1475 

0.9206 

0.0011 

0.9696 

-0.45714 

0.57106 

-0.3851 

1.05078 

1146 

-2 

6 

0.01 

6 

0.1 

150 

45 

4 

9 

1 

0 

5 

1.9556 

0.8854 

0.0086 

0.5 

-0.51356 

0.475886 

-0.05163 

-0.10036 

1147 

2 

6 

0.01 

1 

0.01 

50 

5 

14 

9 

1 

0 

5 

5.3728 

0.682 

0.005 

0.4901 

-0.42561 

-0.07407 

-0.21169 

-0.12463 

1148 

-2 

6 

0.1 

6 

0.1 

150 

45 

14 

9 

1 

0 

5 

2.8588 

1 

0.0015 

0 

-0.49031 

0.785743 

-0.36731 

-1.32603 

1149 

-2 

14 

0.1 

1 

0.01 

50 

5 

4 

1 

1 

0 

5 

27.5488 

0.8758 

0.013 

0.3873 

0.145125 

0.449929 

0.14401 

-0.37663 

1150 

-2 

6 

0.1 

6 

0.01 

150 

45 

4 

1 

1 

0 

5 

3.4235 

0.8509 

0.0041 

0.5249 

-0.47578 

0.382604 

-0.25171 

-0.03932 

1151 

2 

14 

0.1 

1 

0.01 

50 

45 

4 

9 

1 

0 

5 

3.6509 

0.5932 

0 

1 

-0.46992 

-0.31417 

-0.43401 

1.125301 

1152 

-2 

14 

0.1 

6 

0.01 

50 

45 

14 

9 

1 

0 

5 

1.2034 

1 

0 

1 

-0.53291 

0.785743 

-0.43401 

1.125301 

1153 

-2 

14 

0.1 

6 

0.1 

50 

5 

14 

1 

1 

0 

5 

6.0738 

0 

0 

0 

-0.40757 

-1.91807 

-0.43401 

-1.32603 

1154 

2 

6 

0.01 

1 

0.01 

50 

45 

14 

1 

1 

0 

5 

18.3003 

0 

0 

0 

-0.0929 

-1.91807 

-0.43401 

-1.32603 

1155 

2 

6 

0.1 

6 

0.1 

150 

45 

4 

1 

1 

0 

5 

1.3944 

1 

0 

1 

-0.528 

0.785743 

-0.43401 

1.125301 

1156 

-2 

14 

0.1 

6 

0.01 

150 

5 

14 

9 

1 

0 

5 

7.7444 

0.9428 

0.0013 

0.9652 

-0.36457 

0.631085 

-0.37621 

1.039994 

1157 

-2 

14 

0.01 

1 

0.1 

50 

45 

14 

9 

1 

0 

5 

4.7741 

0.8368 

0.0022 

0.7067 

-0.44102 

0.34448 

-0.33619 

0.406326 

1158 

-2 

14 

0.01 

6 

0.01 

50 

5 

4 

9 

1 

0 

5 

6.0885 

0.8079 

0.0005 

0.9051 

-0.40719 

0.26634 

-0.41178 

0.89267 

1159 

2 

14 

0.1 

6 

0.1 

50 

45 

4 

1 

1 

0 

5 

1.6914 

1 

0.0015 

0 

-0.52035 

0.785743 

-0.36731 

-1.32603 

1160 

-2 

6 

0.01 

6 

0.1 

50 

45 

14 

1 

1 

0 

5 

4.0797 

0 

0.0019 

0 

-0.45889 

-1.91807 

-0.34953 

-1.32603 

1161 

-2 

6 

0.1 

1 

0.1 

50 

45 

4 

1 

1 

0 

5 

3.069 

0.75 

0.0018 

0.6647 

-0.4849 

0.109789 

-0.35397 

0.303371 

1162 

2 

14 

0.1 

6 

0.01 

50 

5 

14 

1 

1 

0 

5 

65.197 

0.954 

0.0788 

0.0645 

1.114059 

0.661367 

3.069674 

-1.16792 

1163 

-2 

6 

0.01 

1 

0.01 

50 

5 

4 

1 

1 

0 

5 

2.9437 

0.6351 

0.0011 

0.7402 

-0.48812 

-0.20088 

-0.3851 

0.488446 

1164 

-2 

6 

0.1 

6 

0.1 

50 

5 

4 

9 

1 

0 

5 

1.6278 

0.9505 

0.0012 

0.9522 

-0.52199 

0.651904 

-0.38065 

1.008127 

1165 

2 

14 

0.1 

1 

0.01 

150 

45 

14 

1 

1 

0 

5 

99.773 

0.966 

0.1035 

0.0704 

2.003926 

0.693813 

4.16791 

-1.15345 

1166 

2 

14 

0.01 

1 

0.1 

50 

5 

14 

1 

1 

0 

5 

6.3367 

0.3362 

0.0028 

0.3278 

-0.4008 

-1.00905 

-0.30951 

-0.52248 

1167 

-2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

5 

3.9025 

0.6218 

0.0017 

0.6016 

-0.46345 

-0.23684 

-0.35842 

0.148692 

1168 

2 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

5 

2.3871 

1 

0.0541 

0 

-0.50245 

0.785743 

1.971439 

-1.32603 

1169 

0 

6 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

5 

20.7215 

1 

0 

1 

-0.03059 

0.785743 

-0.43401 

1.125301 

1170 

0 

14 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

5 

2.4 

1 

0 

1 

-0.50212 

0.785743 

-0.43401 

1.125301 

1171 

0 

10 

0.01 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

5 

4.125 

0.9428 

0.0017 

0.9539 

-0.45772 

0.631085 

-0.35842 

1.012294 

1172 

0 

10 

0.1 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

5 

77.0674 

0.918 

0.0443 

0.1103 

1.419562 

0.56403 

1.535702 

-1.05565 

1173 

0 

10 

0.055 

1 

0.055 

100 

25 

9 

5 

1 

0 

5 

2.426 

0.5628 

0.005 

0.8045 

-0.50145 

-0.39637 

-0.21169 

0.646066 

1174 

0 

10 

0.055 

6 

0.055 

100 

25 

9 

5 

1 

0 

5 

7.4101 

1 

0 

1 

-0.37318 

0.785743 

-0.43401 

1.125301 

1175 

0 

10 

0.055 

3.5 

0.01 

100 

25 

9 

5 

1 

0 

5 

3.5792 

1 

0 

1 

-0.47177 

0.785743 

-0.43401 

1.125301 

1176 

0 

10 

0.055 

3.5 

0.1 

100 

25 

9 

5 

1 

0 

5 

2.1946 

1 

0 

1 

-0.5074 

0.785743 

-0.43401 

1.125301 

1177 

0 

10 

0.055 

3.5 

0.055 

50 

25 

9 

5 

1 

0 

5 

1.7267 

1 

0 

1 

-0.51945 

0.785743 

-0.43401 

1.125301 

1178 

0 

10 

0.055 

3.5 

0.055 

150 

25 

9 

5 

1 

0 

5 

1.7626 

0.9681 

0.0891 

0.1119 

-0.51852 

0.699491 

3.527643 

-1.05172 

1179 

0 

10 

0.055 

3.5 

0.055 

100 

5 

9 

5 

1 

0 

5 

122.834 

0.9637 

0.0007 

0.93 

2.597436 

0.687594 

-0.40288 

0.953708 

1180 

0 

10 

0.055 

3.5 

0.055 

100 

45 

9 

5 

1 

0 

5 

4.6935 

0 

0 

0 

-0.44309 

-1.91807 

-0.43401 

-1.32603 

1181 

0 

10 

0.055 

3.5 

0.055 

100 

25 

4 

5 

1 

0 

5 

2.1869 

1 

0 

1 

-0.5076 

0.785743 

-0.43401 

1.125301 

1182 

0 

10 

0.055 

3.5 

0.055 

100 

25 

14 

5 

1 

0 

5 

3.601 

0.1578 

0 

1 

-0.47121 

-1.49141 

-0.43401 

1.125301 

1183 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

1 

1 

0 

5 

2.4475 

0.9561 

0.0056 

0.8669 

-0.5009 

0.667045 

-0.18502 

0.799029 

1184 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

9 

1 

0 

5 

2.3741 

0.9598 

0.0263 

0.5921 

-0.50278 

0.677049 

0.735368 

0.125404 

1185 

0 

10 

0.055 

3.5 

0.055 

100 

25 

9 

5 

1 

0 

5 

2.4704 

1 

0.0075 

0 

-0.50031 

0.785743 

-0.10054 

-1.32603 
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Table  A. 5 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  5)  in  the  Crossed  Array  Design 


A-20 


Crossed  Array  Inner  Array  Design  Points  and  Responses  (Block  6) 


A-21 


9|  5|  l|  0|  6|  5.6766|  0.3362|  0.0019|  0.4202|  -0.41779|  -1.00905|  -0.34953|  -0.29598| 

9|  5|  l|  0|  6|  3.4593 1  0.6218|  0.0019|  0.5759|  -0.47486|  -0.23684|  -0.34953|  0.085693| 

9|  5|  l|  0|  6|  114.7745|  0.9312|  0.0054|  0.5828|  2.390013|  0.59972|  -0.1939l|  0.102607| 

4|  9|  0|  0|  6|  98.534|  0.9624|  0.0342|  0.1818|  1.972038|  0.684079|  1.086626|  -0.88038| 

6|  0.01|  150 1  5 

1|  0.01 1  50 1  5 

4|  1|  0|  0|  6|  2.471l|  0.9583 1  0.1394|  0.0753|  -0.50029|  0.672994|  5.76413l|  -1.14144| 

6|  0.01|  150 1  5 

4|  9|  0|  0|  6|  26.4869|  0.8758|  0.0146|  0.3612|  0.117796|  0.449929|  0.21515l|  -0.4406l| 

1|  0.01 1  50 1  5 

6|  0.01|  150 1  5 

6|  0.01|  50 1  5 

1|  0.01 1  50 1  5 

9|  5|  0|  0|  6|  3.10661  0.8826|  0.0018|  0.9474|  -0.48393|  0.468315|  -0.35397|  0.99636l| 

9|  5|  0|  0|  6|  103.7558|  0.8455|  0.0004|  0.9444|  2.106429|  0.368003|  -0.41622|  0.989007| 

1524  0  10  0.055 

6  0.055  100  25 

9  5  0  0  6  3.4308  0.4572  0.0001  0.9937  -0.47559  -0.68189  -0.42956  1.109857 

1525  0  10  0.055 

3.5  0.01  100  25 

9  5  0  0  6  29.1986  0.7611  0.0052  0.6199  0.187585  0.139801  -0.2028  0.193551 

1526  0  10  0.055 

3.5  0.1  100  25 

9  5  0  0  6  113.7897  0.9751  0.0488  0.1459  2.364667  0.718418  1.735785  -0.96838 

1527  0  10  0.055 

3.5  0.055  50  25 

9  5  0  0  6  2.2144  0.9632  0.002  0.9274  -0.50689  0.686242  -0.34508  0.947334 

1528  0  10  0.055 

3.5  0.055  150  25 

9  5  0  0  6  3.8036  0.6667  0.0225  0.2626  -0.46599  -0.11544  0.566409  -0.68231 

1529  0  10  0.055 

3.5  0.055  100  5 

9  5  0  0  6  3.9833  0.4572  0.0001  0.9937  -0.46137  -0.68189  -0.42956  1.109857 

1530  0  10  0.055 

3.5  0.055  100  45 

9  5  0  0  6  26.1505  1  0.0009  0  0.109138  0.785743  -0.39399  -1.32603 

1531  0  10  0.055 

3.5  0.055  100  25 

4  5  0  0  6  2.7438  0.6959  0.0015  0.6913  -0.49327  -0.03649  -0.36731  0.368576 

1532  0  10  0.055 

3.5  0.055  100  25 

14  5  0  0  6  2.0378  1  0  1  -0.51144  0.785743  -0.43401  1.125301 

1533  0  10  0.055 

3.5  0.055  100  25 

9  1  0  0  6  2.2025  0.9944  0.039  0.4552  -0.5072  0.770601  1.300048  -0.21018 

1534  0  10  0.055 

3.5  0.055  100  25 

9  9  0  0  6  1.6853  0.9558  0.0787  0.1194  -0.52051  0.666234  3.065228  -1.03334 

1535  0  10  0.055 

3.5  0.055  100  25 

9  5  0  0  6  2.0605  1  0  1  -0.51086  0.785743  -0.43401  1.125301 

1536  0  10  0.055 

3.5  0.055  100  25 

9  5  0  0  6  1.4975  1  0  1  -0.52535  0.785743  -0.43401  1.125301 

A-22 


A-23 


Table  A. 6 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  6)  in  the  Crossed  Array  Design 


A-24 


Crossed  Array  Inner  Array  Design  Points  and  Responses  (Block  7) 


A-25 


A-26 


A-27 


Table  A.  7 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  7)  in  the  Crossed  Array  Design 


A- 


Crossed  Array  Inner  Array  Design  Points  and  Responses  (Block  8) 


A-29 


9|  5|  l|  0|  8|  2.2192|  0.9468|  0.0577|  0.1598|  -0.50677|  0.6419|  2.131505|  -0.9343| 

9|  5|  l|  0|  8|  4.6703|  0.9428|  0.0017|  0.9539|  -0.44369|  0.631085|  -0.35842|  1.012294| 

6|  0.01|  150 1  5 

1|  0.01 1  50 1  5 

6|  0.01|  150 1  5 

6|  0.01|  50 1  5 

1|  0.01 1  50 1  5 

9  5  0  0  8  1.8634  0.9631  0.0019  0.9304  -0.51593  0.685972  -0.34953  0.954688 

9  5  0  0  8  1.9332  0.8846  0.0111  0.4326  -0.51413  0.473722  0.059531  -0.26558 

2084  0  10  0.055 

6  0.055  100  25 

9  5  0  0  8  1.7085  1  0  1  -0.51991  0.785743  -0.43401  1.125301 

2085  0  10  0.055 

3.5  0.01  100  25 

9  5  0  0  8  4.7936  0  0.0165  0  -0.44051  -1.91807  0.299631  -1.32603 

2086  0  10  0.055 

3.5  0.1  100  25 

9  5  0  0  8  2.2452  1  0  1  -0.5061  0.785743  -0.43401  1.125301 

2087  0  10  0.055 

3.5  0.055  50  25 

9  5  0  0  8  90.9264  0.8829  0.0007  0.9018  1.776245  0.469126  -0.40288  0.88458 

2088  0  10  0.055 

3.5  0.055  150  25 

9  5  0  0  8  5.6802  0.017  0.0144  0.0048  -0.4177  -1.87211  0.206259  -1.31426 

2089  0  10  0.055 

3.5  0.055  100  5 

9  5  0  0  8  6.877  0.3362  0.0042  0.2446  -0.3869  -1.00905  -0.24726  -0.72643 

2090  0  10  0.055 

3.5  0.055  100  45 

9  5  0  0  8  3.6256  0.8496  0.0007  0.9741  -0.47058  0.379089  -0.40288  1.061811 

2091  0  10  0.055 

3.5  0.055  100  25 

4  5  0  0  8  5.4684  0.8329  0.0209  0.6709  -0.42315  0.333935  0.495268  0.318569 

2092  0  10  0.055 

3.5  0.055  100  25 

14  5  0  0  8  2.9821  0.9006  0.0986  0.0513  -0.48714  0.516984  3.950041  -1.20027 

2093  0  10  0.055 

3.5  0.055  100  25 

9  1  0  0  8  114.782  0  0  0  2.390206  -1.91807  -0.43401  -1.32603 

2094  0  10  0.055 

3.5  0.055  100  25 

9  9  0  0  8  2.1941  0.9632  0.002  0.9274  -0.50742  0.686242  -0.34508  0.947334 

2095  0  10  0.055 

3.5  0.055  100  25 

9  5  0  0  8  19.9839  0  0  0  -0.04957  -1.91807  -0.43401  -1.32603 

2096  0  10  0.055 

3.5  0.055  100  25 

9  5  0  0  8  4.3348  0  0  0  -0.45232  -1.91807  -0.43401  -1.32603 

A-30 


MjEEEEl  MtEiljE]  If  "LaTi 

4|  9|  l|  1|  8|  2.6473 1  0.9927|  0.0226|  0.5824|  -0.49575|  0.766005|  0.570855|  0.101626| 

1|  0.01 1  50 1  5 

4|  1|  l|  1|  8|  122.8044|  0.9539|  0.0005|  0.9402|  2.596674|  0.661097|  -0.41178|  0.97871l| 

1)071  MfcQ] 

1|  0.01 1  50 1  5 

4|  1|  l|  1|  8|  7.536 1  0.7892 1  0.0003|  0.948l|  -0.36994|  0.215778|  -0.42067|  0.998077| 

«!**  i*  # 

lEOH^BSKII]  EES  KSKESKESKa! 

2156  0  10  0.055 

3.5  0.1  100  25 

9  5  1  1  8  20.4349  0  0  0  -0.03796  -1.91807  -0.43401  -1.32603 

2157  0  10  0.055 

3.5  0.055  50  25 

9  5  1  1  8  113.9094  0.974  0.031  0.2048  2.367748  0.715444  0.944344  -0.82399 

2158  0  10  0.055 

3.5  0.055  150  25 

9  5  1  1  8  4.0041  0.6066  0.0065  0.3938  -0.46083  -0.27794  -0.145  -0.36069 

2159  0  10  0.055 

3.5  0.055  100  5 

9  5  1  1  8  19.9535  0  0  0  -0.05035  -1.91807  -0.43401  -1.32603 

2160  0  10  0.055 

3.5  0.055  100  45 

9  5  1  1  8  2.0719  0.8194  0.0041  0.6413  -0.51056  0.297434  -0.25171  0.24601 

2161  0  10  0.055 

3.5  0.055  100  25 

4  5  1  1  8  1.8003  1  0  1  -0.51755  0.785743  -0.43401  1.125301 

2162  0  10  0.055 

3.5  0.055  100  25 

14  5  1  1  8  2.234  0.9452  0.0017  0.9337  -0.50639  0.637574  -0.35842  0.962778 

2163  0  10  0.055 

3.5  0.055  100  25 

9  1  1  1  8  6.2667  0  0  0  -0.4026  -1.91807  -0.43401  -1.32603 

2164  0  10  0.055 

3.5  0.055  100  25 

9  9  1  1  8  1.2707  1  0.0535  0  -0.53118  0.785743  1.944761  -1.32603 

2165  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  8  113.0509  0.9107  0.0026  0.7133  2.345653  0.544292  -0.3184  0.422505 

2166  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  8  3.034  1  0  1  -0.4858  0.785743  -0.43401  1.125301 

2167  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  8  2.3008  1  0  1  -0.50467  0.785743  -0.43401  1.125301 

2168  0  10  0.055 

3.5  0.055  100  25 

9  5  1  1  8  2.6724  0.9812  0.0002  0.9942  -0.49511  0.734911  -0.42512  1.111083 

A-31 


Table  A. 8 


Inner  Array  Design  Points  and  Resultant  Responses  for  the  first  Outer 
Array  Design  Point  (Block  8)  in  the  Crossed  Array  Design 


A-32 


Std  Run 

Image 

Std  Mean  Time 

SNR  Time 

Mean  TPF 

SNR  TPF 

Mean  FPF 

SNR  FPF 

Mean  TFP 

SNR  TFP 

1 

I-* 

00 

-0.120822 

-14.78317 

-0.256308 

-12.43445 

-0.29784 

3.8945552 

0.0051361 

-45.35712 

2 

1-* 

oo 

-0.108727 

-19.072 

-0.159173 

-17.44728 

-0.283946 

0.2155395 

0.1989748 

-14.56336 

3 

bo 

-0.071952 

-23.27925 

-0.910225 

-3.277874 

0.0478592 

-28.63455 

-0.589832 

-4.797729 

4 

bo 

0.5567125 

-7.374325 

-0.016006 

-36.04361 

-0.086085 

-17.88543 

0.0634777 

-22.92305 

5 

i— > 

bo 

0.271719 

-13.11194 

-0.75016 

-4.67746 

-0.411221 

20.82756 

-0.255164 

-13.40217 

6 

h-» 

bo 

-0.421674 

15.816219 

-0.561467 

-6.566535 

-0.356198 

9.4372336 

-0.0897 

-21.91003 

7 

1...8 

-0.113529 

-13.91323 

-0.773041 

-4.097923 

-0.269495 

2.2313397 

-0.882551 

2.3469044 

8 

1...8 

0.1841408 

-16.79981 

0.0025826 

-53.29502 

0.2851804 

-15.50121 

-0.247504 

-13.17979 

9 

1...8 

0.7128879 

-4.420438 

-0.175869 

-16.65609 

0.0445245 

-27.65689 

-0.357936 

-9.372078 

10 

1...8 

0.3049397 

-12.10692 

-0.528244 

-7.210588 

0.0339646 

-27.96081 

-0.407729 

-7.297409 

11 

1...8 

0.4923075 

-10.7214 

-0.471126 

-8.569282 

-0.323962 

4.7999095 

-0.367435 

-9.530125 

12 

oo 

0.1263738 

-21.23339 

-0.545785 

-7.679418 

-0.116098 

-16.64382 

-0.295489 

-11.19822 

13 

bo 

0.2718821 

-13.94204 

-0.409851 

-8.674167 

-0.149445 

-9.594966 

-0.380151 

-8.886776 

14 

bo 

0.1330563 

-18.5666 

0.442392 

-1.245468 

0.1251137 

-19.31912 

0.2649768 

-12.33332 

15 

bo 

1.1461809 

-1.520945 

0.6453472 

14.084735 

1.302271 

-1.309817 

-0.372889 

-8.309521 

16 

bo 

-0.182896 

-9.789974 

-0.42266 

-9.410869 

0.3168602 

-14.2595 

-0.53866 

-5.837571 

17 

i-* 

bo 

-0.280678 

-6.516066 

0.2609322 

-11.24814 

-0.171121 

-7.591663 

0.5278816 

-4.178482 

18 

bo 

-0.446063 

11.908163 

0.4026797 

-1.870324 

-0.132772 

-12.38826 

0.3685147 

-7.709012 

19 

i— » 

bo 

-0.480727 

17.370817 

-0.273612 

-12.62524 

-0.339524 

3.6786708 

0.1865343 

-16.52109 

20 

i— » 

bo 

0.2722527 

-12.2445 

0.1388887 

-16.12231 

0.4435797 

-10.80659 

0.0055957 

-45.6148 

21 

bo 

-0.436249 

13.463497 

-0.762225 

-4.887891 

-0.41789 

23.085309 

-0.424673 

-9.340676 

22 

bo 

-0.076149 

-23.0339 

0.0762278 

-24.17018 

0.032853 

-28.38695 

-0.200898 

-15.17479 

23 

bo 

-0.248706 

-7.037777 

-0.04301 

-28.68745 

1.4384388 

-5.462038 

-0.374973 

-9.955777 

24 

1...8 

0.110348 

-23.04475 

-0.787472 

-3.358661 

-0.148334 

-10.2276 

-0.272967 

-10.9712 

25 

oo 

-0.108626 

-18.89886 

-0.304571 

-12.53672 

-0.242817 

-3.574349 

-0.433958 

-7.800511 

26 

bo 

0.4510161 

-12.00091 

-0.031553 

-31.41268 

-0.228367 

-1.391067 

-0.130576 

-18.52022 

27 

bo 

-0.071893 

-22.42029 

0.2688071 

-10.82316 

-0.397326 

14.086404 

0.6416232 

-2.977221 

28 

I-* 

oo 

-0.25963 

-7.343886 

0.2926006 

-5.97063 

0.2596142 

-13.1862 

0.1478952 

-17.86111 

29 

j-* 

bo 

-0.080086 

-21.14941 

-0.318124 

-10.77162 

-0.238927 

-1.965072 

0.1311649 

-18.37031 

30 

bo 

0.7859575 

-6.522046 

-0.021177 

-33.95698 

0.0662002 

-22.54623 

-0.071499 

-22.06301 

31 

bo 

-0.427946 

6.4435396 

0.1488252 

-15.99939 

-0.396215 

12.990468 

0.6084077 

-3.187641 

32 

i-* 

bo 

-0.125836 

-13.19014 

0.3009487 

-9.75713 

0.1834713 

-16.56686 

0.1558008 

-17.20791 

33 

j-* 

bo 

0.6517135 

-7.435433 

-0.210309 

-13.81212 

0.2162628 

-15.8724 

0.0284543 

-30.93577 

34 

i— » 

bo 

-0.306042 

0.8953235 

-0.991948 

-1.446395 

-0.410109 

21.559377 

-0.148501 

-18.56972 

35 

bo 

-0.445407 

7.2107294 

0.0487503 

-28.00733 

-0.373427 

10.406993 

0.0804225 

-23.57109 

36 

bo 

-0.079772 

-20.75356 

0.2657991 

-10.81146 

-0.310624 

10.513394 

-0.127726 

-18.52968 

37 

1...8 

-0.399571 

5.0337205 

0.527089 

3.0886904 

-0.047736 

-20.63434 

0.579053 

-0.956089 

38 

1...8 

-0.242915 

-8.251586 

0.4324893 

4.6022004 

-0.342303 

12.203527 

0.4043653 

-6.081706 

39 

1...8 

-0.142366 

-17.10707 

0.6480172 

11.428854 

0.536952 

-11.9056 

0.0782163 

-23.18476 

40 

1...8 

0.1465953 

-16.4562 

-0.467476 

-8.320389 

-0.133883 

-8.862226 

-0.511788 

-4.670967 

41 

I-* 

oo 

-0.34088 

4.2490279 

-0.471667 

-8.248015 

-0.40344 

18.714272 

0.0233985 

-33.86156 

A-33 


0.1879189 


0.4939729 


0.158273 


0.2593146 


-0.444352 


0.3172971 


-0.059891 


-0.441732 


-0.136652 


-0.430053 


-0.223462 


0.1287879 


-0.128524 


-0.181822 


0.2510487 


0.2219371 


0.6137785 


-0.378421 


0.2200223 


-0.47886 


-0.106451 


0.5022434 


-0.242177 


0.0646618 


-0.511027 


-0.001437 


-0.117383 


-0.18029 


0.3999152 


-0.12759 


0.5453251 


0.2511783 


-0.076418 


-0.018455 


0.306186 


0.304794 


-0.124196 


0.2795484 


0.2488997 


0.1930771 


-0.142148 


-0.116195 


-0.47625 


-0.004488 


-15.32582 

-0.331305 

-12.02286 

0.1873618 

-16.15059 

-0.485405 

-6.752733 

-10.52917 

-0.828976 

-3.105361 

-0.416223 

26.234004 

-0.247197 

-13.45715 

-15.32117 

-0.179891 

-15.78911 

-0.131104 

-11.93704 

-0.021277 

-33.25489 

-13.5201 

0.6473074 

14.991922 

0.6158738 

-8.33564 

-0.36244 

-8.847448 

14.673828 

0.025903 

-31.63234 

-0.066633 

-21.15326 

-0.275602 

-10.86799 

-12.9018 

0.1563621 

-15.18441 

-0.111652 

-12.43528 

0.1335856 

-16.87869 

-24.70132 

-0.669518 

-6.057807 

0.5569603 

-10.58258 

-0.83046 

-0.766721 

8.458295 

0.4358691 

-2.724926 

-0.382876 

13.235762 

0.6643286 

-2.344974 

-17.48936 

0.2712067 

-10.74553 

0.3068561 

-12.27982 

0.0566139 

-25.92167 

15.246446 

-0.074138 

-23.80465 

0.1145537 

-17.15337 

-0.302107 

-11.38779 

-9.492565 

0.1816766 

-13.70618 

0.3051887 

-13.47418 

0.1731133 

-16.3976 

-18.82258 

-0.044092 

-27.13281 

0.1945871 

-13.50677 

-0.141055 

-16.77852 

-13.48438 

-0.401097 

-10.0321 

-0.298952 

3.6080652 

-0.531214 

-5.042316 

-12.69038 

-0.201217 

-15.33016 

0.2457195 

-14.44456 

-0.085011 

-22.11232 

-14.53163 

0.4313064 

-4.329678 

0.3718832 

-14.05013 

0.0965706 

-20.0367 

5.6637864 

-0.294229 

-12.17187 

-0.36787 

11.438199 

0.2091478 

-14.24887 

-15.32824 

0.3677329 

-8.015522 

1.2600311 

-3.206317 

-0.644098 

-1.618825 

-7.655637 

0.0255313 

-33.50105 

-0.079416 

-21.45184 

0.0716896 

-24.31939 

4.6561111 

-0.438443 

-8.908701 

0.2162628 

-14.99236 

-0.480686 

-5.694999 

-15.46693 

0.4291771 

-0.197429 

0.1823597 

-14.32028 

0.0753666 

-22.10533 

25.314155 

0.1158387 

-17.09743 

-0.385099 

14.688509 

0.6644818 

-2.293467 

-19.98561 

0.3958526 

-0.860151 

-0.098869 

-11.53646 

0.3797601 

-6.194706 

-10.26632 

0.0655815 

-23.58609 

-0.15667 

-7.969217 

-0.148011 

-16.16678 

-9.160903 

0.2508943 

-8.399144 

-0.05774 

-22.15115 

-0.216127 

-13.316 

-27.05751 

-0.211492 

-14.22503 

-0.233369 

-3.15381 

-0.065615 

-23.26302 

25.637185 

0.5234389 

5.9794281 

-0.055517 

-24.07101 

0.5925353 

-1.77778 

-57.70107 

0.0495614 

-25.91974 

-0.325074 

8.8340143 

0.1133928 

-18.85635 

-11.24751 

-0.108713 

-20.38863 

-0.331188 

5.8180551 

-0.126868 

-19.03874 

-18.72498 

0.5847479 

10.502241 

-0.310068 

9.25109 

0.5236224 

-4.063443 

-10.24553 

0.2788112 

-10.30615 

0.111219 

-21.75334 

0.0115708 

-39.57612 

-10.01172 

0.0448973 

-26.12543 

1.1644357 

-7.379945 

-0.142189 

-17.93113 

-19.40754 

0.4772712 

4.7193386 

0.025072 

-26.23276 

-0.221489 

-11.16017 

-18.03461 

0.3990972 

-3.777468 

0.5452888 

-9.642687 

0.0613328 

-25.1648 

-8.722055 

-0.416576 

-9.63282 

-0.351752 

12.723577 

-0.392714 

-8.387353 

-12.98869 

0.5456102 

8.6726355 

0.0778717 

-19.45948 

0.2220785 

-10.17865 

-21.43364 

0.6172951 

6.1260324 

0.0506382 

-26.10438 

0.0707091 

-24.02 

-34.94159 

-0.375614 

-10.71467 

-0.222809 

-4.209516 

-0.110383 

-19.39151 

-11.78766 

-0.026348 

-33.03453 

-0.007164 

-39.99965 

-0.299227 

-11.96589 

-13.79076 

-0.83871 

-3.24469 

-0.377874 

11.951469 

-0.290555 

-11.98958 

-17.04067 

0.2302777 

-11.62422 

0.1995891 

-16.15133 

0.1689154 

-16.12719 

-12.00761 

0.1693742 

-10.47296 

0.2095933 

-15.24359 

0.1344129 

-16.62068 

-14.60611 

-0.075929 

-20.80762 

-0.191685 

-5.746049 

0.0345214 

-29.09258 

-14.56075 

-1.023549 

-0.774544 

0.1567935 

-19.21742 

-0.855249 

2.3056578 

-12.85425 

-0.07478 

-23.83824 

-0.353419 

10.916919 

-0.080599 

-22.85489 

-25.86703 

-0.159376 

-16.90427 

-0.330076 

9.7250043 

-0.505659 

-4.952409 

-18.73295 

-0.235015 

-13.13136 

-0.360644 

10.76672 

0.1783223 

-15.10035 

21.906247 

0.1478451 

-15.71504 

0.0800949 

-19.30877 

-0.005742 

-46.21787 

-48.86939 

-0.451185 

-7.097266 

0.4941564 

-12.73359 

-0.201909 

-14.24611 

A-34 


0.1335601 

-18.96826 

0.4968401 

3.1689086 

1.1727725 

-6.899023 

-0.031664 

-30.10741 

-0.459822 

14.25522 

0.2025974 

-13.76882 

-0.358977 

10.521144 

0.3186302 

-10.71967 

-0.387521 

5.2024062 

-0.173368 

-16.52846 

0.0778717 

-20.09718 

-0.27309 

-12.5238 

-0.155228 

-13.84736 

0.3933516 

-1.41305 

-0.063298 

-18.74567 

0.4094825 

-5.883945 

0.6677892 

-8.049106 

-0.122029 

-19.37641 

0.1723556 

-16.06225 

-0.136734 

-18.93164 

-0.46916 

24.325938 

0.4665574 

-0.642025 

0.1823597 

-14.99173 

0.2833004 

-11.31743 

-0.152445 

-14.48346 

0.3477585 

-8.524933 

0.1106632 

-21.70405 

-0.235033 

-13.79601 

0.0616419 

-23.58307 

-0.392039 

-7.863879 

-0.07886 

-17.10557 

-0.079588 

-22.36599 

-0.477704 

19.27145 

0.196311 

-13.14672 

-0.228923 

-6.23921 

0.3712724 

-8.794228 

-0.487968 

21.659059 

0.199826 

-13.67644 

-0.081083 

-19.16146 

-0.001789 

-55.27941 

0.3061651 

-13.89844 

0.179277 

-14.14885 

0.7403701 

-8.374132 

-0.339919 

-9.879831 

0.2711946 

-13.98985 

0.0430723 

-24.32374 

-0.31896 

7.9744494 

0.4287254 

-3.67085 

-0.015735 

-36.28221 

-0.009754 

-39.0558 

-0.054406 

-23.95772 

0.2206077 

-12.11028 

0.3491683 

-11.24504 

0.2384905 

-11.86315 

0.2773994 

-11.67241 

-0.063409 

-23.67233 

0.2606632 

-13.97423 

-0.414312 

-9.722121 

-0.112207 

-16.59496 

-0.301954 

-11.08391 

0.1989541 

-16.18779 

0.7302808 

17.463947 

0.474148 

-7.848859 

-0.132077 

-18.80051 

0.2498851 

-13.66545 

-0.024287 

-33.75775 

0.212928 

-15.96606 

-0.443334 

-7.491729 

-0.003471 

-49.16479 

0.1762014 

-14.27529 

-0.031063 

-23.82119 

-0.33091 

-7.533784 

0.143173 

-15.35331 

-0.491202 

-7.913426 

0.6575578 

-11.84922 

-0.470146 

-6.695224 

0.0682279 

-23.17847 

-0.899714 

-2.529323 

-0.311735 

4.98768 

-0.479553 

-6.277604 

-0.31773 

1.3919019 

0.0027516 

-51.49298 

-0.280611 

0.6013325 

0.2713808 

-10.53091 

0.2543185 

-13.55225 

0.6192215 

8.2621998 

0.8531949 

-6.477589 

-0.288257 

-10.67921 

0.044973 

-27.0761 

-0.290883 

-11.63905 

0.3257528 

-14.66248 

0.0591572 

-25.49508 

0.208723 

-16.63755 

-0.011849 

-40.09745 

-0.312847 

3.6647576 

-0.105174 

-20.72328 

0.4217192 

-9.639573 

0.4288053 

-2.92583 

-0.195576 

-6.914171 

0.3434499 

-7.016921 

-0.459623 

15.39556 

0.2757018 

-10.75083 

-0.265605 

-2.040623 

0.2323128 

-13.09057 

-0.343291 

1.3787031 

0.364049 

-3.143886 

-0.12777 

-10.25157 

0.2295551 

-10.9839 

-0.097853 

-20.85108 

-0.256443 

-14.60568 

-0.312847 

5.3848475 

-0.465764 

-8.192562 

-0.440491 

9.5982673 

-0.097121 

-21.48586 

-0.364535 

10.530815 

-0.22394 

-14.55492 

-0.152622 

-12.25789 

0.2257826 

-12.35726 

0.1078843 

-19.38379 

0.14287 

-18.30736 

0.1090506 

-22.59168 

-0.124666 

-19.33698 

-0.301731 

3.5023455 

-0.065554 

-24.8589 

-0.156936 

-15.31583 

0.4549985 

0.2745957 

-0.05385 

-23.71782 

0.5955382 

-3.642481 

-0.409396 

9.7491513 

-0.653667 

-6.367066 

-0.019947 

-32.60309 

-0.514024 

-6.187708 

0.7407822 

-7.236138 

-0.309438 

-12.7301 

-0.015501 

-35.51097 

-0.320216 

-11.60279 

0.5396244 

-8.490728 

-0.097391 

-22.01279 

0.1156653 

-20.35267 

0.1183261 

-19.95242 

-0.314301 

0.7074327 

-0.130479 

-16.19368 

-0.111652 

-8.589863 

0.1465777 

-15.49885 

0.3653224 

-10.86081 

0.2265937 

-12.2602 

0.5169436 

-10.77031 

-0.007274 

-43.13102 

0.1580043 

-17.91907 

0.246264 

-11.18812 

-0.013833 

-34.02916 

0.3222765 

-8.778353 

0.2676201 

-14.216 

0.2137844 

-12.48661 

0.2707299 

-12.95605 

0.0826593 

-20.5464 

-0.476876 

18.7941 

0.0827508 

-21.30222 

-0.357865 

8.5810564 

0.6408265 

-1.632558 

-0.03002 

-30.70291 

0.3146367 

-9.314189 

0.0873201 

-20.95758 

-0.561029 

-5.812576 

0.0287809 

-33.72342 

-0.463995 

-7.659512 

-0.298396 

-0.661442 

0.1076016 

-19.43074 

0.2714056 

-13.68247 

0.362866 

-8.161974 

0.5658529 

-8.753719 

-0.383675 

-8.947452 

0.5293443 

-7.839588 

-0.333907 

-11.9724 

-0.185571 

-10.24106 

-0.177672 

-15.44178 

-0.400637 

4.6158162 

0.2903362 

-10.01795 

0.0978801 

-21.80194 

-0.00614 

-44.88655 

-0.112279 

-18.02416 

0.3233227 

-9.055556 

0.2579468 

-9.982369 

-0.028754 

-30.75201 

-0.166938 

-14.52686 

0.0758898 

-22.1338 

0.3946705 

-9.62132 

0.1005233 

-20.93396 

-0.244983 

-8.995017 

0.2253432 

-12.15006 

0.2257111 

-11.689 

-0.070089 

-22.09048 

A-35 


-0.119026 

-18.16761 

-0.116757 

-18.52719 

-0.327297 

6.7308161 

0.402343 

-6.497143 

-0.469886 

16.847507 

0.049629 

-25.87793 

0.5736339 

-12.77861 

0.2180645 

-13.72306 

-0.310316 

-0.467954 

-0.546191 

-7.021447 

-0.377874 

13.182105 

-0.310564 

-11.22741 

-0.105956 

-19.74117 

0.4006519 

-1.580323 

1.0188195 

-7.253284 

0.1267525 

-18.40092 

-0.452484 

16.345768 

-0.102528 

-20.85771 

-0.001606 

-54.19186 

-0.142464 

-17.27251 

-0.366405 

4.1551423 

-0.402956 

-9.954101 

-0.185571 

-3.841827 

-0.451699 

-7.548516 

-0.149135 

-15.57058 

0.3788185 

-7.855063 

0.3585443 

-11.35197 

-0.322239 

-10.59286 

0.4166807 

-8.995487 

0.6102314 

7.4673185 

0.1445662 

-16.57563 

0.3829775 

-8.064497 

-0.300325 

-5.533317 

0.314096 

-9.391279 

-0.343971 

7.7423218 

0.3407534 

-10.00346 

-0.352508 

3.5996689 

-0.164953 

-15.80505 

-0.156115 

-10.83568 

0.0894924 

-19.39495 

0.5410332 

-8.401236 

-0.280405 

-13.73706 

-0.362312 

7.8117708 

-0.151688 

-18.40628 

-0.132912 

-17.55602 

0.5852887 

5.7855267 

-0.198354 

-4.911768 

0.3451352 

-8.23692 

-0.14599 

-16.20739 

0.6292595 

4.8738217 

-0.352308 

9.3657921 

0.2918188 

-11.28801 

0.0448649 

-26.92078 

0.1059022 

-18.33929 

-0.220586 

-1.679952 

0.0229082 

-30.84592 

-0.070489 

-23.42528 

0.0428695 

-28.33293 

0.2618373 

-16.45144 

0.0430704 

-27.39141 

-0.327545 

1.1805069 

0.1333459 

-13.67306 

-0.273942 

2.0305475 

0.4488263 

-3.085098 

0.1925238 

-16.56559 

-0.09685 

-21.51594 

0.2551679 

-14.38607 

-0.512523 

-6.288526 

-0.100473 

-18.94336 

-0.529663 

-7.903988 

0.2996308 

-16.45714 

-0.247228 

-13.51235 

0.307384 

-13.26541 

-0.258099 

-14.53338 

-0.392324 

14.860017 

-0.196179 

-15.89599 

-0.268272 

-6.998704 

0.2430532 

-8.386472 

-0.344527 

8.0631986 

0.4087164 

-7.535241 

-0.494133 

23.549183 

0.2351446 

-12.63879 

0.3546538 

-12.55578 

0.2906544 

-11.99058 

0.0082772 

-41.11451 

0.2020904 

-13.07139 

-0.312291 

4.2790348 

0.5751616 

-3.229405 

0.3712187 

-10.32696 

-0.005461 

-46.72236 

0.4652554 

-10.49839 

-0.192288 

-15.39864 

-0.095052 

-21.4208 

0.7454897 

21.566781 

-0.355086 

11.712749 

0.4194104 

-8.352309 

0.1825834 

-15.46167 

0.1721119 

-13.91182 

0.5380636 

-9.158767 

-0.004271 

-45.76394 

0.5236076 

-8.693904 

0.1082004 

-18.82346 

0.3941147 

-11.46772 

0.0939048 

-21.39723 

0.4020948 

-10.65307 

0.2659681 

-10.56886 

-0.271163 

5.7483523 

0.0332651 

-28.91259 

0.2966529 

-13.46531 

-0.460412 

-8.082185 

-0.117765 

-13.30305 

-0.260802 

-12.47148 

-0.117804 

-18.69778 

0.607122 

9.1108019 

0.2045912 

-14.94732 

0.2463773 

-11.78497 

-0.480287 

23.225853 

0.3319749 

-9.014744 

-0.380097 

13.511692 

0.4505116 

-7.750616 

-0.447569 

15.026881 

0.0739296 

-21.36017 

-0.279499 

0.8025775 

0.4029865 

-6.747512 

-0.390224 

5.1847399 

0.0983315 

-19.48109 

-0.306177 

2.0395537 

0.3436031 

-8.510775 

-0.471424 

19.601552 

0.3613113 

-4.489432 

-0.380097 

15.502634 

0.5751003 

-3.530646 

-0.179363 

-10.38454 

-0.155895 

-17.26914 

0.0428571 

-26.00544 

-0.027742 

-31.13955 

-0.386585 

4.6625205 

-0.135075 

-17.49566 

-0.002717 

-43.71988 

-0.27119 

-9.986471 

0.1904729 

-15.48347 

0.1712331 

-14.44005 

-0.332299 

10.266807 

0.0521096 

-25.24771 

-0.269096 

-6.655176 

-0.30937 

-12.69927 

0.7109134 

-9.768021 

-0.53195 

-5.736654 

0.3277104 

-11.97947 

-0.05413 

-26.64127 

-0.327297 

4.0889976 

0.219474 

-13.70102 

0.4956349 

-8.821804 

-0.540884 

-6.663155 

-0.003829 

-46.9264 

-0.437083 

-7.618361 

-0.081161 

-21.58252 

-0.41857 

-8.53357 

0.1023264 

-22.68522 

-0.089363 

-21.72659 

-0.071985 

-23.34861 

0.2207467 

-12.49482 

0.414123 

-13.17391 

0.0867653 

-21.14156 

-0.061807 

-25.27562 

-0.291356 

-13.22435 

-0.260603 

0.8002869 

0.033786 

-30.56259 

0.2681873 

-13.3692 

0.081534 

-21.40979 

-0.302287 

1.7344182 

0.4256919 

-6.76941 

-0.118905 

-18.54703 

0.4337736 

-5.535679 

0.3163045 

-12.93718 

0.3526117 

-9.399474 

0.0086848 

-43.56813 

-0.177728 

-14.91349 

0.1028822 

-18.93258 

-0.005282 

-45.55208 

0.1987459 

-16.17006 

0.4462449 

-1.309605 

0.5241689 

-9.477932 

0.2678571 

-11.81914 

-0.34726 

3.7609807 

-0.008097 

-41.40515 

-0.272274 

0.4075466 

0.2088414 

-13.18001 

0.657158 

-6.811385 

0.1635948 

-14.73996 

0.3896684 

-10.18663 

-0.212174 

-11.37869 
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-0.41784 

12.584432 

-0.084717 

-22.76414 

-0.341748 

7.9950878 

-0.025904 

-32.9212 

0.491685 

-9.01073 

-0.075862 

-23.61612 

-0.419002 

23.993005 

0.4153963 

-8.437959 

-0.029916 

-30.80169 

0.1656565 

-14.79226 

-0.041623 

-26.01829 

0.1874229 

-14.30443 

-0.247116 

-1.433078 

-0.354287 

-11.29861 

-0.181681 

-6.284753 

-0.260159 

-12.29718 

0.2173145 

-16.70211 

-0.625683 

-5.271304 

-0.214472 

-2.18165 

-0.209324 

-13.79763 

-0.281974 

1.7979745 

-0.49715 

-7.570009 

-0.235036 

-2.922977 

-0.205433 

-13.57512 

0.2382454 

-15.09252 

0.1982375 

-13.1972 

-0.255045 

-2.964547 

0.0735587 

-23.38702 

0.9825084 

-2.22371 

-0.111349 

-18.37019 

0.90266 

-9.752956 

-0.052041 

-25.82223 

-0.087008 

-16.5436 

-0.447636 

-7.273203 

-0.341748 

6.8896192 

0.1610098 

-16.2362 

-0.420098 

6.4760116 

0.4166382 

-7.134035 

-0.236704 

-5.188748 

0.3661553 

-9.644568 

0.3165662 

-12.30861 

-0.646198 

-6.537118 

-0.237815 

-6.07616 

-0.664475 

-4.253988 

0.2078644 

-15.3814 

-0.112194 

-20.25885 

-0.011054 

-33.92849 

-0.227219 

-12.19834 

-0.44595 

17.24093 

-0.065249 

-22.74414 

-0.326186 

10.004385 

0.0513742 

-25.77105 

-0.025192 

-32.2649 

-0.724473 

-4.947682 

0.0789833 

-24.04929 

-0.590199 

-4.410275 

-0.039006 

-28.52612 

-0.190199 

-15.29923 

-0.042734 

-25.09742 

-0.33713 

-8.466682 

-0.454688 

9.1573165 

0.0930928 

-20.01384 

-0.396771 

16.082326 

0.5781338 

-3.259211 

-0.23991 

-7.139681 

0.301895 

-9.704562 

-0.265049 

-2.188699 

0.2194127 

-13.53058 

0.2546791 

-13.11167 

-0.158058 

-16.8894 

0.2151512 

-14.06447 

-0.09638 

-20.12819 

0.1764163 

-17.4117 

0.6369653 

12.244286 

0.8315192 

-7.948968 

0.1949607 

-14.19715 

0.365832 

-10.95843 

-0.31809 

-12.40548 

0.2151512 

-16.0048 

-0.551959 

-3.508996 

-0.480491 

27.756453 

0.2419379 

-7.580802 

-0.200578 

-8.88681 

0.6228092 

-2.002365 

0.4710246 

-9.069068 

-0.119596 

-19.48692 

0.5519582 

-5.742193 

-0.545064 

-4.20742 

-0.467743 

21.522689 

0.2921275 

-9.771569 

-0.278944 

1.6397381 

0.3256165 

-10.51027 

-0.294762 

1.545884 

-0.039191 

-29.57201 

-0.366758 

13.642302 

-0.019193 

-35.41177 

0.6604189 

-8.349769 

-0.463454 

-8.581651 

-0.257824 

1.0237684 

-0.329225 

-9.578903 

0.2801715 

-12.58862 

-0.713557 

-4.665072 

-0.073858 

-18.05248 

-0.613089 

-1.662936 

0.6048678 

-8.374107 

0.0051175 

-45.84811 

-0.166675 

-11.9797 

0.3305804 

-9.838774 

-0.274038 

-6.212828 

0.2057406 

-10.51351 

-0.358977 

10.827921 

0.5013153 

-5.119915 

-0.443493 

12.647709 

0.3105134 

-9.567918 

0.8304076 

-5.824706 

-0.076677 

-22.19009 

-0.089561 

-21.75807 

0.1425051 

-16.04009 

-0.151113 

-8.244331 

0.1374771 

-16.98444 

0.1204625 

-19.28185 

-0.297304 

-13.12447 

0.5547372 

-10.4717 

-0.715462 

-3.721854 

0.5234422 

-8.649727 

-0.472681 

-8.16795 

-0.313402 

9.0573405 

-0.317857 

-8.464123 

0.4229732 

-9.228995 

-0.235995 

-13.17102 

-0.323962 

9.8355917 

-0.040397 

-26.70003 

0.16055 

-17.41708 

0.5514234 

4.2344518 

0.6742314 

-4.697108 

-0.05774 

-22.4046 

0.9612243 

-4.327798 

0.6517687 

9.2938078 

0.0489708 

-25.46332 

0.1344742 

-17.07627 

-0.076679 

-23.14948 

0.3148733 

-3.677258 

0.8759822 

-7.17011 

-0.074471 

-22.21377 

0.265308 

-13.80881 

-0.147141 

-17.69998 

-0.210026 

0.4764717 

-0.355577 

-8.093321 

-0.353901 

3.7626502 

-0.316332 

-10.43615 

-0.31896 

8.9962608 

-0.103151 

-20.30224 

-0.328328 

1.3169936 

0.5220194 

6.428969 

0.753709 

-5.367878 

-0.211377 

-11.20701 

-0.059892 

-25.35345 

0.4197138 

-7.058675 

-0.356754 

11.525223 

0.1201952 

-20.04929 

0.3949976 

-10.04596 

-0.044464 

-28.12215 

0.2468311 

-14.98471 

0.0153397 

-37.22498 

-0.084237 

-22.17872 

-0.004988 

-47.59825 

-0.230034 

-2.495083 

-0.344791 

-10.13445 

0.1667715 

-17.2816 

0.2075656 

-12.57312 

-0.017724 

-35.11607 

0.3413356 

-9.13444 

-0.08108 

-21.96042 

-0.088536 

-21.97488 

-0.411777 

23.502899 

0.6665654 

-1.943839 

-0.456657 

15.016311 

-0.192903 

-15.63859 

-0.3612 

7.8196557 

0.0723024 

-24.33862 

0.2811254 

-14.03595 

0.2740119 

-10.30822 

-0.208359 

-4.668309 

0.2653751 

-11.54731 

0.1797762 

-16.93404 

0.1321968 

-17.4767 

-0.381208 

14.716575 

0.3069864 

-10.88297 

-0.094518 

-20.58719 

-0.018778 

-34.87196 

-0.170009 

-5.650705 

0.0836092 

-19.74235 
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-0.408784 

5.3567494 

0.1800881 

-14.57544 

0.0695349 

-18.53036 

0.1868407 

-13.36061 

-0.03695 

-28.00802 

0.145851 

-11.97647 

0.1540146 

-17.38472 

0.0644276 

-23.16882 

-0.451833 

15.576436 

-0.096039 

-21.84959 

-0.330076 

7.5204348 

0.3655118 

-9.245209 

-0.160954 

-14.82128 

0.3434324 

-8.744793 

-0.040511 

-26.59547 

-0.22489 

-14.07487 

0.185886 

-14.59615 

-0.603917 

-5.642229 

-0.284502 

1.2009302 

-0.248423 

-12.29896 

-0.401852 

11.850021 

-0.608818 

-5.725069 

-0.373427 

11.982641 

-0.103948 

-20.55845 

-0.202683 

-8.926598 

0.5730539 

6.6426394 

-0.082195 

-19.43115 

0.3220008 

-10.40622 

-0.215829 

-7.95827 

-0.533888 

-7.24363 

-0.397326 

16.779125 

0.0348891 

-30.62968 

0.1660116 

-16.4257 

0.2536657 

-11.0135 

0.4441355 

-12.49098 

0.0412012 

-28.6546 

0.2547132 

-14.26783 

-0.016175 

-37.29381 

-0.384543 

12.713875 

0.2576534 

-12.37778 

0.3286617 

-8.765557 

-0.513441 

-7.304437 

0.2062586 

-14.0206 

-0.516905 

-5.098429 

-0.422726 

8.7953959 

0.1135067 

-18.52338 

0.119 

-17.45341 

0.210496 

-12.22806 

-0.14577 

-15.46021 

0.1929312 

-13.14986 

-0.186127 

-6.196791 

-0.073153 

-22.2918 

-0.328019 

1.6390832 

0.2787774 

-10.51106 

1.0910718 

-5.567295 

-0.135141 

-17.91046 

-0.375373 

5.3376656 

-0.052744 

-26.88552 

-0.192797 

-2.762624 

0.1386721 

-17.36956 

0.0342274 

-29.01705 

-0.080458 

-23.1444 

-0.32952 

9.1295226 

0.1475888 

-16.86186 

0.1171701 

-20.16476 

0.5758253 

4.3295089 

0.2890709 

-12.8693 

0.5237756 

-2.013765 

-0.149744 

-16.12836 

0.2719164 

-7.531356 

-0.347305 

6.1482923 

0.972154 

13.243688 

0.1785887 

-16.85253 

0.2596479 

-6.88688 

0.3285318 

-9.965164 

0.0739264 

-20.70653 

-0.485942 

23.758453 

0.2465006 

-11.92858 

-0.048292 

-25.02377 

-0.001391 

-58.52947 

-0.456751 

17.468872 

-0.157449 

-16.6477 

-0.420114 

23.598707 

0.7917057 

-0.686278 

0.3156773 

-12.81497 

-0.62673 

-6.881857 

-0.423448 

23.557377 

-0.121659 

-20.49638 

0.0226826 

-33.20638 

0.6107721 

7.895342 

0.0967685 

-19.03368 

0.1183874 

-19.40647 

0.6149775 

-7.754997 

-0.106043 

-20.67878 

0.5308383 

-9.468437 

-0.255808 

-12.43129 

-0.398879 

8.4342496 

-0.452537 

-9.537265 

-0.038288 

-27.51424 

-0.584439 

-5.599131 

-0.380086 

4.425926 

-0.314879 

-10.98438 

-0.277832 

0.7033851 

0.2154293 

-13.94162 

-0.356996 

3.2861585 

0.3094319 

-9.605389 

0.2907382 

-9.833716 

-0.083663 

-21.7304 

-0.248481 

-6.595534 

0.094073 

-16.94611 

-0.192797 

-2.93958 

0.2887853 

-6.46375 

0.6467293 

-7.418567 

-0.77811 

-2.816677 

-0.120544 

-14.11097 

-0.226177 

-13.86049 

0.3160392 

-12.2557 

0.4963331 

8.5876527 

-0.346194 

14.253877 

0.6698134 

3.6317793 

-0.488284 

25.939813 

0.3093305 

-1.408637 

-0.300619 

4.7388799 

0.7356316 

4.2945773 

-0.171614 

-10.90534 

0.0464858 

-26.17149 

0.5625182 

-9.629834 

0.0164122 

-36.79639 

-0.031588 

-29.82148 

-0.152109 

-16.7728 

0.0128446 

-35.33385 

-0.039938 

-28.33564 

0.5768539 

-7.0326 

0.4681796 

-0.118558 

-0.27672 

1.6037059 

0.6313889 

0.3797224 

-0.180294 

-11.59175 

-0.135109 

-18.35029 

-0.063854 

-22.2372 

-0.186772 

-14.64143 

-0.381091 

5.2490264 

-0.555181 

-6.564308 

0.7742731 

-5.746908 

-0.784743 

-0.462079 

1.1439357 

-2.582865 

0.2134126 

-12.65913 

0.2629489 

-10.75072 

0.0135625 

-37.12626 

-0.113285 

-19.06922 

-0.001879 

-56.06509 

-0.145555 

-10.57487 

-0.260128 

-11.41382 

0.2449462 

-14.57082 

0.1060712 

-16.25773 

-0.315626 

9.6539626 

0.4469572 

-3.209502 

0.7314337 

-6.981026 

-0.204631 

-14.99074 

0.0895433 

-23.29066 

-0.26975 

-12.52962 

-0.461456 

22.103434 

-0.190199 

-13.46684 

-0.228923 

-2.151492 

0.2170227 

-12.5168 

0.6808295 

-6.266688 

-0.328432 

-12.13898 

-0.358977 

9.2117743 

-0.076248 

-23.11804 

-0.006258 

-44.51239 

0.1300675 

-17.45664 

-0.341192 

8.9437052 

0.5511386 

-3.32806 

0.3466895 

-10.16128 

0.5640637 

12.19732 

0.6342147 

-11.70637 

0.3848773 

-6.333376 

0.3920769 

-11.36958 

-0.129972 

-17.18774 

-0.068856 

-21.00055 

0.1339227 

-17.83636 

-0.48837 

30.297874 

0.6148278 

8.6756182 

0.134562 

-17.86376 

0.1155377 

-19.93024 

-0.23132 

-7.74555 

-0.188712 

-13.86903 

-0.295061 

2.4520601 

0.0713219 

-22.54093 
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Appendix  B.  Additional  Figures  and  Analysis 
from  the  Combined  Array  Design 

The  following  Appendix  contains  additional  figures,  analysis,  and  results. 
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Table  B.l  Combined  Array  Control  Levels  and  Raw  Responses 
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Standardized  Total  Response  = 

M  [1] 

M[2] 

M[3] 

M  [4] 

M[5] 

M  [6] 

M[7] 

M[8] 

0.928732608 

0.112710268 

0.117218679 

0.1262355 

0.112710268 

0.117218679 

0.108201857 

-0.16093589 

*  A 

-0.019531055 

-0.020312297 

-0.021874781 

-0.019531055 

-0.020312297 

-0.018749812 

-0.019531055 

-0.021093539 

0.287762309 

*  B 

0.03492261 

0.036319515 

0.039113324 

0.03492261 

0.036319515 

0.033525706 

0.03492261 

0.037716419 

0.18357479 

*  C 

0.022278494 

0.023169634 

0.024951913 

0.022278494 

0.023169634 

0.021387354 

0.118276589 

*  D 

0.014353955 

0.014928113 

0.01607643 

0.014353955 

0.014928113 

0.013779797 

0.622239684 

*  E 

0.075514525 

0.078535106 

0.084576268 

0.075514525 

0.078535106 

0.072493944 

0.047485806 

*  F 

0.00576284 

0.005993354 

0.006454381 

0.00576284 

0.005993354 

0.005532327 

0.00576284 

0.006223868 

-0.059166689 

*  G 

-0.007180423 

-0.00746764 

-0.008042074 

-0.007180423 

-0.00746764 

-0.006893207 

-0.007180423 

-0.007754857 

0.021716595 

*  H 

0.002635509 

0.002740929 

0.00295177 

0.002635509 

0.002740929 

0.002530089 

0.002635509 

0.00284635 

*  J 

0.044374362 

0.046149337 

■ili».U!:hL-U:W 

*  K 

0.009266845 

0.009637519 

0.010378867 

0.010008193 

-0.160423161 

*  L 

-0.01946883 

-0.020247583 

-0.02180509 

-0.01946883 

-0.020247583 

-0.018690077 

-0.01946883 

-0.021026337 

HgBBl 

0.774824531 

U.'.IHB 

0 

0 

-1.779421789 

0 

-0.224587216 

0 

0 

0 

0 

0 

0 

-0.224587216 

2.456865236 

0 

0 

0.333942848 

0 

0 

0 

0 

0 

0.333942848 

1.464838509 

0 

0 

0 

0.177771664 

0 

0 

0 

0 

1.22827625 

0 

0 

0 

0 

0 

0 

0 

0.155025158 

-2.906496946 

0 

-0.28650865 

0 

0 

0 

-0.29846376 

*  BE 

-0.03622133 

-0.037670183 

-0.04056789 

-0.03622133 

-0.037670183 

-0.034772477 

-0.03622133 

-0.039119036 

-0.145106798 

ram 

-0.017610048 

0 

0 

-0.609422884 

(aurciHi 

0 

0 

0 

0.050601077 

glMIEll 

0 

0 

0 

1.104654031 

0 

0 

0.391108084 

0 

0 

0 

0 

0 

0 

0 

-0.258359334 

0 

0 

0 

0 

0 
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Table  lays  out  steps  for  developing  the  Mean  Model  from  the  General 
Model 


| Standardized  Total  Response  =  1 

1.091525694 

-0.16093589 

*  A 

0.373480157 

*  B 

0.18357479 

*C 

0.118276589 

*  D 

0.493179787 

*  E 

0.047485806 

*  F 

-0.059166689 

*  G 

0.021716595 

*  H 

0.32925413 

*J 

0.076358805 

*  K 

-0.160423161 

*  L 

-0.29846376 

*  BE 

-1.197693658 

*  JA2 

Figure  B.2  Reduced  Mean  Model 
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Appendix  C.  Code  for  AutoGAD  and  Iterative 

Master  Function 

The  following  Appendix  contains  MATLAB®  code  for  executing  the  AutoGAD 
algorithm  as  well  as  code  for  the  master  function  used  to  feed  parameters  to  and 
record  responses  from  AutoGAD.  It  should  be  noted  that  the  AutoGAD  code 
was  modified  slightly  to  enable  recording  the  results  of  hundreds  of  sequential  repli¬ 
cations. 
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%This  is  a  modified  AutoGAD  script  for  use  in  RPD  experiments  where  a 
%parameter  matrix  is  used  to  feed  multiple  runs  of  AutoGAD  iteratively  and 
%record  the  results  for  analysis 

%Mods  by:  Capt  Matt  Davis 
%Jan  2009 

%AutoGAD  version  1.0  % 


%Hyper spectral  Autonomous  Global  Anomaly  Detector  (AutoGAD)  % 

%Using  FastICA  % 

%Author :  Capt  Robert  Joseph  Johnson  % 

%Feb  2008  % 

9'ici^icicicieiciciciricicicici(icicici(ic^^i(ici(icicicir^ic'k-k-k'k'k-k-k'k'k-k-k'kic'k-k'k-k'k'k'k-k'k'k-k-kic'k'k-k'k'k-k-k'k'k-k-k9' 


%Tactical  Decisions  By  User - 

f unctn=params ( 10 ) ; %ob jective  function  in  ICA  to  use.  Options  [l=tanh,  2=pow3] 
orthogonalization=params ( 11 ) ; %f ind  ICs  in  parallel  (symm)  or  one  by  one  (delf ) . 

%Options  [ symm=l ,  defl=2] 

dim_ad justment=params ( 1 ) ; %how  much  to  adjust  max  distance  log  scale  secant  line  (MDLS) 
%dimensionality  decision 

max_score_thresh=params ( 2 ) ; %threshold  above  which  decision  is  made  to  declare  target 
bin_width_SNR=params ( 3 ) ; %bin  width  when  using  zero-detection  histogram  method  to 
%determine  breakpoint  between  background  and  potential  targets  for 
%calculating  potential  target  SNR  (PT  SNR) 

PT_SNR_thresh=params ( 4 ) ; %2 ; %threshold  above  which  decision  is  made  to  declare  target 
bin_width_ident=params ( 5 ) ; %bin  width  when  using  zero-detection  histogram  method  to 
%determine  breakpoint  between  background  and  targets  for  identifying  target 
%pixels  from  selected  target  signals 

threshold_both_sides=params ( 12 ) ; %l=identif iy  outliers  on  both  sides  of  IC  signal, 
%0=identify  ouliers  on  side  with  highest  magnitude  scores  only 

clean_sig=params ( 13 ) ; %0  =  no  signal  smoothing,  1  =  signal  smoothing  prior  to  target 
%identif i cat ion 

smooth_iter_high=params ( 6 ) ; %number  of  iterations  to  complete  for  iterative  smoothing 
%of  low  SNR  object 

smooth_iter_low=params ( 7 ) ; %20 ; %number  of  iterations  to  complete  for  iterative  smoothing 
%of  high  SNR  object 

low_SNR=params ( 8 ) ; %Threshold  decision  for  choosing  smooth_iter_low  or  smooth_iter_high 
window_size=params ( 9 ) ; %image  window  size  for  smoothing 
show_plots=2 ; %l=yes,  2=no 


switch  num2str ( f unctn) 
case  ' 1 ' 

f unct= ' tanh ' ; 
case  ' 2 ' 

funct= ' pow3 ' ; 

end 

switch  num2str (orthogonalization) 
case  ' 1 1 

orthog= ' symm ' ; 
case  ' 2 ' 

orthog= ' def 1 ' ; 

end 
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switch  num2str (params ( 14 ) ) 


case  ' 1 ' 

tempi 

= 

' ARES1C '  ; 

temp  3 

= 

' ARESlC_mask ' ; 

case  ' 2 ' 

tempi 

= 

'ARES ID ' ; 

temp  3 

= 

' ARESlD_mask ' ; 

case  ' 3 ' 

tempi 

= 

' ARES1F ' ; 

temp  3 

= 

' ARESlF_mask ' ; 

case  ' 4 ' 

tempi 

= 

' ARES2C '  ; 

temp  3 

= 

' ARES2C_mask ' ; 

case  ' 5 ' 

tempi 

= 

' ARES2D ' ; 

temp  3 

= 

' ARES2D_mask ' ; 

case  ' 6 ' 

tempi 

= 

' ARES2F ' ; 

temp  3 

= 

' ARES2F_mask ' ; 

case  ' 7 ' 

tempi 

= 

' ARES3F ' ; 

temp  3 

= 

' ARES3F_mask ' ; 

case  ' 8 ' 

tempi 

= 

' ARES4F ' ; 

temp  3 

= 

' ARES4F_mask ' ; 

end 

% - Solicit  User  Input  to  Load  HSI  Image  File - 

%display ( ' This  program  requires  the  Image  Processing  Toolbox  for  MATLAB.1); 

%di splay ( 'Make  sure  your  version  of  MATLAB  has  this  toolbox.'); 

%display ( '  ' ) ; 

%display ( ' Make  sure  you  have  in  your  working  directory  the  all  the  files  for') 
%display ( ' FastICA  and  the  Center_and_PCA . m  file'); 

%di splay ( '  ' ) ; 

%display ( ' The  first  several  lines  in  the  AutoGAD  algorithm  detail  default'); 
%display (' settings  for  AutoGAD.  If  you  would  like  to  experiment'); 

%display (' changing  these  settings,  hit  Ctrl  c  to  interrupt  this  run.  Open'); 
%display('up  AutoGAD  in  the  the  editor  and  make  changes.'); 

%di splay ( '  ' ) ; 

%display (' Please  hit  enter'); 

%display ( '  ' ) ; 

%answer=input (  '  '  )  ; 

%display (' Enter  you  image  cube  file  name  to  be  processed.'); 

%display ( ' File  should  be  in  .mat  format  '); 

%display ( '  ' ) ; 

%di splay (' !Make  sure  to  put  it  in  single  quotes!') 

%display ( ' ! Make  sure  the  image  cube  is  in  the  same  directory  as  this  code!'); 
%display ( '  ' ) ; 

%templ=input ( ' ' ) ; 
temp2=struct2cell (load (tempi ) ) ; 
im_cube=temp2 { 1 } ; 

%di splay ( '  ' ) ; 

%display  ('Enter  truth  mask'); 

%display ( '  ' ) ; 

%display('If  you  do  not  have  a  truth  mask  and  this  is  a  real  target  search'); 
%display ( ' with  no  truth  knowledge,  enter  O'); 

%display ( '  ' ) ; 

%temp3=input ( ' ' ) ; 
if  temp3~=0; 

temp4=struct2cell (load (temp3 ) ) ; 
truth=temp4 { 1 } ; 

end 
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clear  tempi 
clear  temp2 
clear  temp4 
%clc; 

%di splay ( 1  '  )  ; 

%display (' Please  enter  the  good  bands  for  this  HSI  sensor' ); 

%display (' These  are  the  bands  that  are  NOT  the  atmospheric  absorption  bands'); 
%di splay ( '  ' ) ; 

%display('If  this  the  the  210  band  HYDICE  sensor,  LtCol  Tim  Smetek  concluded'); 
%display( 'that  the  good_bands  =  [5:72,  78:85,  92:99,  116:134,158:199]'); 

%di splay ( '  ' ) ; 

%display('If  this  is  HYDICE  data  and  you  would  like  to  keep  these  bands,  type') 
%display('l  and  hit  enter'); 

%di splay ( '  ' ) ; 

%display('If  this  is  not  HYDICE  data  or  you  do  not  want  to  keep  those  bands'); 
%display ( ' just  hit  enter  and  then  enter  the  bands  you  wish  to  keep'); 

%di splay ( '  ' ) ; 

answer=l;  %input ( ' ' ) ; 
if  answer==l 

good_bands= [5:72, 78:85,92:99, 116 : 134, 158:199] ; 

else 

good_bands=input ( ' good_bands  =  ' ) ; 

end 


% - Ask  User  if  they  want  to  see  color  image - 

%display ( '  ' ) ; 

%display('Do  you  want  to  see  a  RGB  image  of  your  HSI  file?'); 

%di splay ( '  ' ) ; 

%display('If  so,  enter  1.  If  not  just  hit  enter.  '); 

%di splay ( '  ' ) ; 

answer=  2 ;  %input  (  '  '  ) ; 
if  answer==l 

Red=input (' Please  enter  the  band  number  for  red,  HYDICE  is  50  ') ; 

display ( '  ' ) ; 

Green=input (' Please  enter  the  band  number  for  green,  HYDICE  is  29  '); 

display ( '  ' ) ; 

Blue=input (' Please  enter  the  band  number  for  blue,  HYDICE  is  22  '); 

R=im_cube ( : , : , Red) ; 

G=im_cube ( : , : , Green) ; 

B=im_cube ( : , : , Blue) ; 

%Borrowed  from  Lt  Col  Tim  Smetek,  lines  142  -  163,  offer  a  way  to  make 
%an  RGB  image  look  better.  The  following  lines  are  used  in  conjunction 
%with  the  mat2gray  function  to  perform  a  2%  linear  stretch  on  the  image 
%data 

ml=size (R, 1 ) ; 
n=size (R, 2 ) ; 

low_id=f loor ( 0 . 02*ml *n ) ; 
hi_id=f loor ( 0 . 98*ml*n) ; 

r_vec=reshape (R, ml*n, 1 ) ; 
r_vec=sort (r_vec) ; 
r_vec=double (r_vec) ; 
min_R=r_vec (low_id) ; 
max_R=r_vec (hi_id) ; 

g_vec=reshape (G, ml*n, 1 ) ; 
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g_vec=sort (g_vec) ; 
g_vec=double (g_vec)  ; 
min_G=g_vec (low_id) ; 
max_G=g_vec  (hi_id)  ; 

b_vec=reshape (B, ml*n, 1 ) ; 
b_vec=sort (b_vec) ; 
b_vec=double (b_vec)  ; 
min_B=b_vec  (low_id)  ; 
max_B=b_vec  (hi_id)  ; 

%The  IPT  function  mat2gray  to  scales  the  values  in  each  matrix  between  0 
%and  1.  This  is  necessary  because  the  matrices  are  of  type  double  and 
%imshow  requires  double  value  matrices  to  be  scaled  between  0  and  1 

R=mat2gray (double (R) , [min_R  max_R] ) ; 

G=mat2gray (double (G) , [min_G  max_G] ) ; 

B=mat2gray (double (B) , [min_B  max_B] ) ; 

%**Now  stack  the  three  matrices  into  a  3D  array  and  display  the  image 

RGB=cat (3,R,G,B) ; 

figure  (1) 

imshow(RGB, [] ) ; 

title ('True  Color  Image'); 

impixelinf o; 

%**Turn-on  the  interactive  pixel  value  utility 
clear  RGB 
clear  RGB 

clear  r_vec  g_vec  b_vec 

end 


tic; 

% - Resize  Image  Cube  into  matrix  where  each  row  is  a  pixels - 

% - signature  in  the  spectral  bands - 

dims=size ( im_cube, 3 ) ; 

num_pixels=size ( im_cube, 1 ) *size ( im_cube, 2 ) ; 
num_lines=size ( im_cube, 1 ) ; 
num_col=size (im_cube, 2) ; 

%**Place  all  the  pixel  vectors  into  a  single  matrix  where  each  row 

%corresponds  to  a  pixel  vector 

data_matrix=zeros (num_pixels, dims ) ; 

data_matrix_truth=zeros (num_pixels,  1) ; 

for  x=l:dims 

data_matrix ( : , x) =reshape ( im_cube ( : ,  : ,  x)  ,  num_pixels, 1 ) ; 

end 

clear  im_cube; 

%If  HSI  cube  is  too  large  for  MATLAB  since  MATLAB  converts  variables  to 
%double  precision,  this  will  make  file  smaller  so  that  MATLAB  can 
%operate  on  it . 
if  num_pixels*dims  >  25*10A6 

data_matrix=single (data_matrix) ; 

end 


if  temp3~=0; 

data_matrix_truth=reshape (truth, num_pixels, 1) ; 

end 
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% - Keep  bands  that  are  not  atmospheric  absorption  bands 

data_matrix_new=data_matrix ( : , good_bands) ; 
dims=size (data_matrix_new, 2 ) ; 
clear  data_matrix; 


% - Set  negative  pixel  values  =  0  (remove  bad  pixels) 

[m, n]  =  size (data_matrix_new) ; 
for  i  =1 : m 

for  j=  1 : n 

if  data_matrix_new ( i, j )  <  0 
data_matrix_new ( i, j )  =  0; 

end 

end 

end 


% - Perform  PCA - 

[Ac,  Lc,  TotVarCompC,  YscorC]  =Center_and_PCA  (data_matrix_new)  ; 

%Function  written  by  Capt  Johnson  does  PCA  on  covariance  matrix 

Lplot=diag (Lc) ; 

%checks  for  eigenvalues  10A-4  and  smaller  and  moves  the  endpoint  of  the 
%eigenvalue  curve  to  the  point  where  eigenvalues  are  greater  than  10A-4 
%so  that  the  MDSL  method  in  the  next  section  is  not  biased  by  pathological 
%cases  where  the  endpoint  of  the  log  scale  eigenvalue  curve  has  extremely 
%small  endpoints  and  grossly  alters  the  theoretical  shape  of  the  curve  that 
%should  arise  for  eigenvalues  of  covariance  matrices  of  spectral  data 
%that  follow  the  LMM 
while  Lplot (dims ) <=10 A-4 ; 
dims=dims-l ; 

end 

L=logl0 (Lplot) ; 
clear  data_matrix_new; 
clear  Ac; 
clear  Lc; 


% - Dimensionality  Assessment - 

%slope  of  line  connecting  endpoints  of  scree  plot  of  eigenvalues 
m_slope  =  ( L ( 1 ) -  L (dims ) ) / ( 1-dims ) ; 

%calculate  Euclidean  distances  from  scree  plot  curve  to  line  connecting 

%endpoints 

Eqdist= [ ] ; 

for  i=l:dims 

x_int=(L(i)  -  L ( 1 ) +  m_slope  +  ( i/m_slope) ) / (m_slope  +  (l/m_slope) ) ; 
y_int=L(l)+  m_slope* (x_int-l ) ; 

Eqdist (i) =sqrt ( (i-x_int) A2+ (L (i) -y_int) A2) ; 

end 

%find  the  point  on  the  log  scale  eigencurve  curve  with  the  largest  distance 

%from  the  line  connecting  the  endpoints 

[max_Eqdist,  index_dim] =max (Eqdist ) ; 

reduced_dim  =  index_dim; 

k=reduced_dim-l ; 

k=k+dim_ad justment ; 

percent_var=TotVarCompC (k, 1) ; 

Y=YscorC ( : , 1 : k) ; 
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clear  YscorC; 
if  show_plots==l 
figure (3 )  ; 

semilogy (Lplot (1 :dims) , 

title ({'Plot  of  Eigenvalue  vs.  PC  Component',... 

sprintf ( ' Dimensionality  =  %i ' , k) } , ' f ontweight ' , ' b ' ) ; 

end 


% - Perform  ICA  on  reduced  PCA  space - 

[icasig.  A,  W] =fastica (Y ',' approach  1 , orthog,  'g',  funct,  'epsilon',... 

.00001,  ' stabilization ',' on ' ,  ' verbose ',' of f ') ; 

icasig=icasig ' ; 

%If  an  IC  score  has  a  high  signals,  make  them  always  positive 
for  j=l : k 

if  abs (min ( icasig ( : ,  j ) ) ) >max ( icasig ( : , j ) ) 
icasig (:, j ) =-icasig ( : , j ) ; 

end 

end 

clear  Y 


% - Find  the  Kurtosis  of  Each  Signal - 

kurt=zeros (k, 1 ) ; 
for  j=l : k 

kurt ( j ) =abs (kurtosis ( icasig (:, j ))) ; 

end 

%this  statistic  will  not  be  used  in  AutoGAD,  but  is  included  if  the  user 
%wishes  to  compare  this  value  to  the  PT  SNR  and  max  pixel  score  values 


% - Find  the  Max  Score  of  Each  Signal 

maxim=zeros (k, 1) ; 
for  j=l : k 

maxim ( j ) =max (icasig ( : , j ) ) ; 

end 


% - Find  the  PT  SNR  of  each  signal - 

for  j=l : k 

bins= [ ] ; 
f req= [ ] ; 

bins=min (icasig(:,j)): bin_width_SNR: max ( icasig (:, j )) ; 
freq=hist (icasig (:, j ), bins) ; 
count=l ; 

%find  the  bin  that  is  at  center  (mean)  of  the  ICA  signal 
for  i=l : size (bins , 2 ) 
if  bins ( i ) <0 

count=count+l ; 

else 

break 

end 

end 

%from  the  center  of  the  signal  keep  counting  until  the  first  zero 
%bin  is  found 
countl=l ; 

for  i=count+l : size ( f req, 2 ) 
if  freq(i)>  0; 
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countl=countl  +  l ; 


else 

break 

end 

end 

%if  there  are  no  zero  bins  until  the  very  end  of  the  tail  then  the 
%threshpoint  is  set  above  the  last  data  point 
if  count+countl  >  size (bins, 2) 

thresh_pt ( j ) =max (icasig ( : , j ) ) ; 

else 

%otherwise  use  the  first  bin  were  a  zero  value  occurs 
thresh_pt ( j ) =bins (count+countl ) ; 

end 

end 

PT_SNR=zeros (k, 1 ) ; 
for  j=l : k 

potent_target= [ ] ; 
potent_bkrd= [ ] ; 

%find  the  indices  of  those  pixels  greater  than  threshold 
ind  =  icasig (:, j ) >thresh_pt ( j ) ; 

%store  those  pixels  greater  than  threshold  in  vector 
potent_target=icasig ( ind, j ) ; 
if  size (potent_target, 1 ) ==0 
potent_target=0 ; 

end 

%find  the  indices  of  those  pixels  less  than  threshold 
ind2  =  icasig (:, j ) <=threshjpt ( j ) ; 

%store  those  pixels  less  than  threshold  in  vector 
potent_bkrd=icasig ( ind2, j ) ; 
power_target ( j ) =var (potent_target ) ; 
power_bkrd ( j ) =var (potent_bkrd) ; 

PT _ SNR ( j ) =10*logl0 (power_target ( j ) /power_bkrd ( j ) ) ; 

end 


one=ones (num_pixels , 1 ) ; 


% - Plot  Abundance  Maps  from  ICs  Frames  with  PT  SNR  and  Max  Pixel  Score - 

if  show_plots==l 
figure  (4) 
d=ceil (sqrt (k) ) ; 
for  j = 1 : k 

subplot (d, d, j ) 

ICsig(:, : ,  j ) =re shape (icasig(:,j), num_lines, num_col ) ; 

ICsig_grey ( : ,  : , j ) =mat2gray (double ( ICsig ( : ,  : , j ) ) )  ; 
imshow ( ICsig_grey ( : ,  : ,  j  )  )  ; 

if  maxim ( j ) >=max_score_thresh  &&  PT_SNR ( j ) >=PT_SNR_thresh 

title ({ sprintf (' Map  %i  \n  SNR  %4.3f  \n  Max  Score  %4 . 3f 1 , j , PT_SNR ( j ) 
, maxim ( j ) ) , ' Potential  Target ' } , ' f ontweight ' , ' b ' ) ; 

else 

title ({ sprintf (' Map  %i  \n  SNR  %4.3f  \n  Max  Score  %4 . 3f 1 , j , PT_SNR ( j ) 
, maxim ( j ) ) , ' Non-Target ' } , ' f ontweight ' , ' b ' ) ; 

end 

end 

clear  ICsig; 
clear  ICsig_grey; 


figure (5) 


Plot  IC  signals 


C-8 


PT_SNR_line= [ ] ; 
for  j=l : k 

PT_SNR_line ( : , j ) =thresh_pt ( j ) *one; 

end 

for  j=l : k 

subplot (d, d, j ) 

plot ( icasig ( : , j ) , ' . '  ,  ' MarkerEdgeColor '  , '  r  '  )  ; 

hold  on 

plot (PT_SNR_line ( : , j ) , ' LineWidth ' , 2 ) ; 

xlabel ( ' Pixel ' ) ; 

ylabel ( 'Abundance  (IC  Score)'); 

if  maxim ( j ) >=max_score_thresh  &&  PT_SNR ( j ) >=PT_SNR_thresh 

title ({ sprintf (' Map  %i  \n  SNR  %4.3f  \n  Max  Score  %4 . 3f ' , j , PT_SNR ( j ) 
, maxim ( j ) ) , ' Potential  Target ' } , ' f ontweight ' , ' b ' ) ; 

else 

title ({ sprintf (' Map  %i  \n  SNR  %4.3f  \n  Max  Score  %4 . 3f ' , j , PT_SNR ( j ) 
, maxim ( j ) ) , ' Non-Target ' } , ' f ontweight '  , ' b ' ) ; 

end 

axis ( [0, num_pixels, -10, 30]); 

end 

clear  PT_SNR_line 


end 


% - Keep  only  Those  Signals  Above  Both  Thresholds - 

ind_max= [ ] ; 
ind_SNR= [ ] ; 
ind_both= [ ] ; 

ind_max  =  maxim>=max_score_thresh; 
ind_SNR  =  PT_SNR>=PT_SNR_thresh; 
ind_both=ind_max+ind_SNR; 

[rind,cind]=  f ind ( ind_both==2 ) ; 
if  size (rind, 1 ) ==0 

display ('NO  TARGETS') 
target_sig=zeros  (num__pixels,  1)  ; 
target_vec=zeros (num_pixels, 1) ; 

else 

target_sig=icasig ( : , rind) ; 

end 

clear  icasig; 

num_tgt_maps=size (target_sig,  2 )  ; 
for  j=l : num_tgt_maps 

tgt_sig_map ( : ,  : ,  j ) =reshape (target_sig ( : , j ) , num_lines, num_col ) ; 

end 


target_sig_clean= [ ] ;  %Place  here  to  make  available  outside  if  statement  — mtd 
if  size (rind, 1 ) ~=0 

% - Show  Abundance  Maps  of  Retained  Target  Signals - 

if  show_plots==l 

d=ceil (sqrt (num_tgt_maps ) ) ; 
tgt_gray= [ ] ; 
figure ( 6 ) 

for  j=l : num_tgt_maps 
subplot (d, d, j ) ; 

tgt_sig_map_gray ( : ,  : , j ) =mat2gray (tgt_sig_map ( : ,  : , j  )  )  ; 
imshow (tgt_sig_map_gray ( : , : ,  j  )  )  ; 

title ({ sprintf (' Map  %i  \n  SNR  %4.3f  \n  Max  Score  %4 . 3f ' , rind ( j ) , . . . 
PT_SNR (rind ( j ) ) , maxim (rind ( j ) ) ) , ' Potential  Target '},... 

' f ontweight ' , ' b ' ) ; 

end 

clear  tgt_sig_map_gray 
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end 

clear  target_sig 


% - Clean  (IAN  Filtering) Target  Signals  prior  to  Identification - 

if  clean_sig==l 

for  j=l : num_tgt_maps 

if  PT _ SNR (rind ( j ) ) <low _ SNR 

for  c=l : smooth_iter_high 

[tgt_sig_map  (  : ,  : ,  j  )  ]  =wiener2  (tgt_sig_map  j  )  , 

[window_size, window_size] ) ; 

end 

else 

for  c=l : smooth_iter_low 

[tgt_sig_map  (  : ,  : ,  j  )  ]  =wiener2  (tgt_sig_map  j  )  , 

[window_size, window_size] ) ; 

end 

end 

end 


- Plot  IAN  Filtered  Target  Maps - 

if  show_plots==l 

for  j=l : num_tgt_maps 

clean_map_gray ( : ,  : , j ) =mat2gray (tgt_sig_map ( : ,  : , j  )  )  ; 

end 

figure ( 7 ) 

for  j=l : num_tgt_maps 
subplot (d, d, j ) ; 

imshow (clean_map_gray ( : ,  : ,  j  )  )  ; 

title (sprintf ( 'Filtered  Map  %i',rind(j)),' f ontweight ' , ' b ' ) ; 

end 

end 

end 


Identify  Target  Pixels  from  Selected  Target  Maps 


%Move  from  527  to  470 
%target_sig_clean= [ ] ; 

for  j=l : num_tgt_maps 

target_sig_clean ( : , j ) =re shape (tgt_sig_map ( : , : , j ) ,  num_pixels,  1 ) ; 

end 

for  j=l : num_tgt_maps 
bins= [ ] ; 
f req= [ ] ; 

bins= (min (target_sig_clean ( : , j ) ) ) : bin_width_ident : . . . 

max (target_sig_clean ( : , j ) ) ; 
freq=hist (target_sig_clean ( : , j ) , bins ) ; 
count=l ; 

%find  the  bin  that  is  at  center  (mean)  of  the  ICA  signal 
for  i=l : size (bins , 2 ) 
if  bins ( i ) <0 

count=count+l ; 

else 

break 

end 

end 
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%from  the  center  of  the  signal  keep  counting  until  the  first  zero 
%bin  is  found 
countl=l ; 

for  i=count+l : size ( f req, 2 ) 
if  freq(i)>  0; 

countl=countl+l ; 

else 

break 

end 

end 

%if  there  are  no  zero  bins  until  the  very  end  of  the  tail  then  the 
%threshpoint  is  set  above  the  last  data  point 
if  count+countl  >  size (bins, 2) 

thresh_pt_ident ( j ) =max (target_sig_clean ( : , j ) ) + . 01 ; 

else 

%otherwise  use  the  first  bin  were  a  zero  value  occurs 
thresh_pt_ident ( j ) =bins (count+countl ) ; 

end 

end 

target=zeros (num_pixels,  num_tgt_maps ) ; 
for  j=l : num_tgt_maps 

ind_tgt  =  target_sig_clean ( : , j ) >thresh_pt_ident ( j ) ; 
target ( : , j ) =ind_tgt ; 

end 

target_vec=zeros (num_pixels, 1) ; 
for  j=l : num_tgt_maps 

target_vec=target_vec  +  target (:, j ) ; 

end 

%end 


%Checks  both  sides  of  the  selected  target  signals  for  target  pixels  if  user 
%specified  this  option 
if  threshold_both_sides==l 

target_sig_clean_left=-target_sig_clean; 
for  j=l : num_tgt_maps 
bins= [ ] ; 
f req= [ ] ; 

bins= (min (target_sig_clean_left ( : ,  j ) ) )  : bin_width_ident : . . . 

max (target_sig_clean_left (  : ,  j )  )  ; 
freq=hist (target_sig_clean_left ( : , j ) , bins ) ; 
count=l ; 

for  i=l : size (bins , 2 ) 
if  bins ( i ) <0 

count=count+l ; 

else 

break 

end 

end 

countl=l ; 

for  i=count+l : size ( f req, 2 ) 
if  freq(i)>  0; 

countl=countl+l ; 

else 

break 

end 

end 

if  count+countl  >  size (bins, 2) 

thresh_pt_ident_left ( j ) =max (target_sig_clean_left ( : , j ) ) + . 01 ; 

else 
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end 


thresh_pt_ident_left ( j ) =bins ( count+count 1 )  ; 

end 


target_lef t=zeros (num_pixels,  num_tgt_maps ) ; 
for  j=l : num_tgt_maps 

ind_tgt  =  target_sig_clean_left ( : , j ) >thresh_pt_ident_left ( j ) ; 
target_lef t ( : , j ) =ind_tgt ; 

end 

target_vec_lef t=zeros (num_pixels, 1) ; 
for  j=l : num_tgt_maps 

target_vec_left=target_vec_left  +  target_left ( : , j ) ; 

end 

target_vec=target_vec+target_vec_left ; 

end 


target_pic  =  reshape (target_vec, num_lines, num_col) ; 


end 

% - Plot  Target  Signals  with  Calculated  Thresholds - 

if  show_plots  ==1 

if  size (rind, 1 ) ~=0 

d=ceil ( sqrt (num_tgt_maps ) ) ; 
linetrh_ident= [ ] ; 
for  j=l : num_tgt_maps 

linetrh_ident ( : , j ) =thresh_pt_ident ( j ) *one; 

end 

if  threshold_both_sides==l 
for  j=l : num_tgt_maps 

linetrh_ident_left ( : , j ) =-thresh_pt_ident_lef t ( j ) ‘one; 

end 

end 

figure ( 8 ) 

for  j=l : num_tgt_maps 
subplot (d, d, j ) 

plot (target_sig_clean ( : , j ) , ' . ' ,  'MarkerEdgeColor ' , ' r ' ) ; 
hold  on 

plot ( linetrh_ident ( : , j ) , ' LineWidth ' , 2 ) ; 
if  threshold_both_sides==l 
hold  on 

plot ( linetrh_ident_lef t ( : , j ) , ' LineWidth ' , 2 ) ; 

end 

xlabel ( 1  Pixel ' ) ; 

ylabel ( 'Abundance  (IC  Score)'); 

title (( sprintf (' Map  %i  \n  SNR  %4.3f  \n  Max  Score  %4 . 3f ' , rind ( j ) , . . . 
PT_SNR (rind ( j ) ) , maxim (rind ( j ) ) ) , ' Potential  Target 
' f ontweight ' , ' b ' ) ; 
axis ( [0, num_pixels, -10, 30]); 

end 

clear  linetrh_ident 

end 

clear  one 

end 


% - Grade  Performance  of  AutoGAD  if  Truth  Mask  was  Provided- 

if  temp3~=0; 

% - Confusion  Matrix  Calculation - 
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ConfusMat= [ ] ; 

ConfusMat (1, 1) =0;  %(TP) 

ConfusMat (1, 2) =0;  %(FP) 

ConfusMat ( 2 , 1 ) =0 ;  %(FN) 

ConfusMat (2, 2) =0;  %(TN) 

for  i=l : num_pixels 

if  target_vec ( i, 1 ) >=  1  &&  data_matrix_truth ( i, 1 )  >=  1 
ConfusMat (1,1) =Conf usMat ( 1 , 1 )  +1 ; 

else 

if  target_vec ( i, 1 ) >=  1  &&  data_matrix_truth ( i, 1 )  ==  0 
ConfusMat (1,2) =ConfusMat ( 1 , 2 ) +1 ; 

else 

if  target_vec (i, 1 ) ==  0  &&  data_matrix_truth ( i , 1 )  ==  1 
ConfusMat (2,1) =ConfusMat (2, 1 ) +1; 

else 

if  target_vec (i,  1 ) ==  0  &&  data_matrix_truth ( i , 1 )  ==  0  I 
ConfusMat (2,2) =ConfusMat ( 2 , 2 ) +1 ; 

end 

end 

end 

end 

end 

APER  =  ( ConfusMat ( 1 , 2 ) +ConfusMat ( 2 , 1 ))/( num_pixels ) ; 

TPF  =  ConfusMat (1, 1) / (ConfusMat (1, 1) +ConfusMat (2, 1 )) ; 

FPF  =  ConfusMat ( 1, 2 )/ (ConfusMat ( 1, 2 ) +ConfusMat (2, 2 )) ; 

Perc_tgt  =  ConfusMat ( 1 , 1 )/ (ConfusMat ( 1, 1 ) +Conf usMat ( 1, 2 )) ; 


- Show  Target  Locations  to  the  User - 

target_vec_color=zeros (num_pixels, 1) ; 
for  i=l : num_pixels 

if  target_vec ( i , 1 ) >=1  &&  data_matrix_truth (i, 1 ) >=1 
target_vec_color (i, 1) =4; 

elseif  target_vec (i, 1 ) >=1  &&  data_matrix_truth (i,  1 ) ==0 
target_vec_color (i, 1 ) =2 ; 

end 

end 

target_pic_color  =  uint8 (reshape (target_vec_color , num_lines, num_col ) ) ; 
if  size (rind, 1 ) ~=0 
figure ( 9 ) 

imshow (mat2gray (target_pic_color ) ) ; 
colormap ( ' Hot ' ) 

title ( sprintf ( 1 TPF  =  %4.6f  \n  FPF  =  %4.6f  \n  Percent  TGT  =  %4.6f', 
TPF,  FPF, Perc_tgt ) , ' f ontweight ' , '  b '  )  ; 
impixelinf o ; 
elseif  size (rind, 1 ) ==0 
figure ( 9 ) 

imshow (target_pic) ; 

title ('No  Targets  Detected') 

end 

figure  (2) 
imshow (truth, [] ) ; 
title ( ' Truth  Mask ' ) ; 
impixelinf o; 

else 

if  size (rind, 1 ) ~=0 
figure ( 9 ) 

imshow (target_pic) 
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%  title ({' Suspected  Target  Pixels'}); 

%  impixelinf o ; 

%  elseif  size (rind, 1 ) ==0 

%  figure (9) 

%  imshow(target_pic) 

%  title ({'No  Targets  Detected'}); 

%  impixelinf o; 

%  end 

end 

time=toc; 

newres  =  [time, TPF, FPF, Perc_tgt] ; 

load  RPDresults;  %loads  the  results  matrix  stored  in  RPDresults .mat 

results  =  [results;  newres];  %adds  the  current  runs  results  to  the  results  matrix 

save  RPDresults  results;  %saves  the  modified  results  matrix  before  it  is  cleared  for  th 

next  run 

clear  all; 
close  all; 
clc; 


Published  with  MATLAB®  7.4 
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function  []  =  RPDFunc (params ) 

%This  function  iteratively  calls  a  modified  AutoGAD  to  test  multiple 
%settings  for  the  algorithm  in  a  single  execution.  The  results  of  AutoGAD 
%are  stored  in  the  results  matrix  which  is  iteratively  saved  to 
% ' RPDre suits .mat 1 

%Capt  Matt  Davis 
%Jan  2009 


[r,c]  =  size (params ) ;  %obtain  dimensions  of  Design  Matrix 

for  i  =  l:r  %runs  AutoGAD  using  the  parameters  in  RPDmatrix  and  saves  results 
to  'results' 

load  RPDparams; 
params  =  params(i,:); 
i  %displays  iteration 

RPD_AutoGAD ; 

%load  RPDresults;  %loads  the  results  matrix  stored  in  RPDresults .mat 
%results  =  [results;  newres] ;  %adds  the  current  runs  results  to  the  results  matrix 
%save  RPDresults  results;  %saves  the  modified  results  matrix  before  it  is  cleared 
for  the  next  run 


end 

%phonetext ; 


end 


Published  with  MATLAB®  7.4 


The  code  above  controlled  the  passing  of  parameters  to  AutoGAD  and  record¬ 
ing  output  during  testing. 
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Appendix  D.  Blue  Dart 

Its  nighttime  in  Iraq  and  the  darkness  conceals  an  insurgent  hard  at  work  on  a 
nondescript  roadside.  As  he  concludes  his  work  planting  an  improvised  explosive 
device  (IED),  he  takes  great  pains  to  cover  all  evidence  of  his  handiwork.  Early  the 
next  morning  an  Unmanned  Aerial  Vehicle  (UAV)  armed  to  the  teeth  with  cameras 
and  sensors  making  its  rounds  at  50,000  feet  captures  a  hyperspectral  image  of  the 
piece  of  road  where  IED  lies  in  wait. 

Recent  research  efforts  proposed  several  techniques  to  automatically  detect 
anomalous  objects  like  IEDs  within  hyperspectral  images.  However,  most  new  al¬ 
gorithms  are  insufficiently  verified  to  be  robust  to  the  broad  range  of  data  being 
made  available.  While  finding  ships  in  the  ocean  is  one  thing,  locating  tanks  under 
camouflage  netting  at  the  edge  of  a  tree  line  or  well  disguised  roadside  bombs  is  quite 
another. 

The  problem  of  noise  (variation  in  inputs  or  operating  environment)  and  the 
need  for  systems  to  be  robust  regardless  of  it,  reaches  far  beyond  software  on  a  sensor 
platform.  Consistent  performance  regardless  of  operating  conditions  is  something  all 
weapon  systems  should  be  designed  for. 

General  Patton  said,  “A  good  plan,  violently  executed  now,  is  better  than  a 
perfect  plan  next  week.”  By  extension,  always  finding  80%  of  the  ships,  tanks,  and 
IEDs  regardless  of  time  of  day  or  location  is  preferable  to  finding  all  the  ships,  tanks, 
and  IEDs,  but  only  when  the  conditions  are  just  right. 

Researchers  at  AFIT  apply  techniques  known  as  Robust  Parameter  Design 
(RPD)  to  determine  optimal  system  settings  that  maximize  performance  while  min¬ 
imizing  output  variance.  The  goal  of  this  research  is  to  improve  upon  the  speed, 
accuracy,  and  robustness  of  automated  systems  to  locate  anomalies  quickly  and  con¬ 
sistently  regardless  of  the  characteristics  of  the  image  presented  to  it. 
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By  leveraging  the  power  of  RPD  during  the  design  phase  of  the  detection  algo¬ 
rithm,  the  AFIT  researchers  expect  to  improve  algorithm  performance  and  deliver 
an  algorithm  to  the  warfighter  that  can  locate  anomalies  within  a  scene  quickly  and 
consistently. 

The  use  of  RPD  is  widespread,  including  everything  from  packaging  toothpaste 
to  high  speed  fluid  dynamics.  While  future  engagements  will  continue  to  require  the 
development  of  specialized  weapons,  RPD  can  ensure  those  weapons  are  consistently 
effective  wherever  and  whenever  the  engagement  occurs. 

Back  at  50,000  feet,  onboard  image  processing  hardware  executes  a  few  lines 
of  code  that  quickly  scan  the  acres  big  image  of  the  earth  below  and  in  a  matter 
of  seconds  passes  on  to  a  user  the  exact  location  of  a  spot  on  the  side  of  the  road 
that  doesnt  look  quite  like  it  should.  Despite  the  insurgents  efforts,  the  UAV  spots 
that  somethings  amiss  with  a  particular  stretch  of  road  and  an  Explosive  Ordinance 
Disposal  (EOD)  team  is  dispatched  to  investigate  and  safely  dispose  of  the  IED. 

The  most  amazing  part  of  this  scenario  is  not  the  sensor  laden,  but  pilotless 
aircraft;  or  even  the  fearless  EOD  team;  but  the  1000  or  so  lines  of  code  sitting  in 
a  computer  at  50,000  feet  that  instantly  identified  something  suspicious  within  the 
collected  image.  This  little  piece  of  software  wouldve  been  easily  fooled  if  not  for  the 
training  via  RPD  it  received  during  its  development. 
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